Using Large-scale Contrastive Language-Image Pre-training to Maximize Brain MRI-Based Headache Classification
Fazle Rafsani1, Devam Sheth2, Yiming Che1, Jay Shah1, Md Mahfuzur Rahman Siddiquee1, Catherine Chong2, Simona Nikolva2, Gina Dumkrieger2, Baoxin Li1, Teresa Wu1, Todd Schwedt2
1Arizona State University, School of Computing and Augmented Intelligence, 2Mayo Clinic
Objective:
To optimize headache detection accuracy and biomarker extraction from relatively small brain MRI dataset.
Background:
Deep learning methods showed promise for classifying headache disorders and extracting biomarkers from MRI data. Pre-trained models perform better for specialized tasks when fine-tuned on small, domain-specific datasets. Our model is fine-tuned on a pre-trained model to optimize headache classification and biomarker extraction.
Design/Methods:
We leveraged BioMedCLIP, a contrastive language-image model, pre-trained on PMC-15M (dataset of 15 million biomedical image-text pairs), to maximize headache classification. We fine-tuned BioMedCLIP using relatively small MRI dataset from 528 Healthy Controls (HC, including 424 from public IXI dataset), 96 participants with migraine (Mig), 49 with persistent post-traumatic headache (PPTH), and 48 with acute post-traumatic headache (APTH). All T1-weighted images were registered to MNI-152 1mm template. Six participants from each cohort were used for validation and testing, rest were used for fine-tuning. To enhance classification, we employed a novel evaluation method aggregating slice data and performing patient-level predictions using multi-instance learning, capturing the likelihood of disease manifestation from slices indicating headache biomarkers. For biomarker extraction, we utilized GradCAM, a deep learning explainability technique, to identify the brain regions significantly associated with each headache phenotype.
Results:
We evaluated three models on three classification tasks: HC vs. Mig, HC vs. APTH, and HC vs. PPTH, achieving accuracies of 91.67%, 83.33%, and 91.67% on held-out test sets, respectively. Key regions included superior frontal cortex, middle temporal white matter, and rostral middle frontal areas for Mig; inferior parietal, cerebellar cortex, and superior parietal cortex for APTH; and left cerebellar cortex, pars triangularis cortex, and middle temporal cortex for PPTH.
Conclusions:
Fine-tuning BioMedCLIP model on relatively small neuroimaging dataset maximizes headache classification and biomarker extraction. This approach provides a robust framework for classifying headache disorders and identifying relevant biomarkers from limited MRI data.
Disclaimer: Abstracts were not reviewed by Neurology® and do not reflect the views of Neurology® editors or staff.