Expert Accuracy and Inter-rater Agreement of “Must-know” EEG Findings for Adult and Child Neurology Residents
Fabio Nascimento1, Roohi Katyal2, Marcia Olandoski3, Hong Gao4, Samantha Yap5, Rebecca Matthews6, Stefan Rampp7, William Tatum8, Roy Strowd9, Sandor Beniczky10
1Washington University Medical School, 2Louisiana State University Health Shreveport, 3Pontifícia Universidade Católica do Paraná, 4Wake Forest University School of Medicine, 5Massachusetts General Hospital, 6Emory University, 7University Hospital Erlangen, Department of Neurosurgery, 8Mayo Clinic, 9Wake Forest School Of Medicine, 10Aarhus University & Danish Epilepsy Center
Objective:
To study the accuracy and inter-rater agreement (IRA) of “must-know” routine EEG (rEEG) findings among international experts.
Background:
Accurate and reliable EEG interpretation is critical in the care of patients with seizures and epilepsy. We recently published a list of “must-know” rEEG findings for trainees based on expert opinion. Subsequently, we used this curricular framework to develop and validate an online, multiple-choice rEEG examination for trainees. This study builds upon our prior work by assessing the performance of "must-know" EEG findings among experts.
Design/Methods:
A previously validated online rEEG examination was disseminated to EEG experts. It consisted of a survey and 30 multiple-choice questions predicated on the previously published “must-know” rEEG findings divided into four domains: normal, abnormal, normal variants, and artifacts. Questions contained de-identified 10-to-20-second epochs of EEG that were considered unequivocal examples by five EEG experts.
Results:
The examination was completed by 258 international EEG experts. Overall mean accuracy and IRA (AC1) were 81% and substantial (0.632), respectively. Domain-specific mean accuracies and IRA were: 76%, moderate (0.558) (normal); 78%, moderate (0.575) (abnormal); 85%, substantial (0.678) (normal variants); 85%, substantial (0.740) (artifacts). Academic experts had a higher accuracy than private practice experts (82% vs. 77%; p=0.035). Country-specific overall mean accuracies and IRA were: 92%, almost perfect (0.836) (U.S.); 86%, substantial (0.762) (Brazil); 79%, substantial (0.646) (Italy); 72%, moderate (0.496) (India).
Conclusions:
Collective expert accuracy and IRA of “must-know” rEEG findings are suboptimal and heterogeneous. We recommend development and implementation of pragmatic, accessible, country-specific ways to measure and improve expert accuracy and IRA.