High Inter-Rater Reliability in Consensus Diagnoses and Overall Assessment in the Asian Cohort for Alzheimer’s Disease Study
Yara Alkhodair1, Ging-Yuek Hsiung1, Boon Lead Tee2, Pei-Chuan Ho4, Phoenix Au Yeung1, Wai Haung Yu5, Guerry Peavy6, Victor Henderson7, Yun-Beom Choi8, Dolly Reyes-Dumeyer9, Haeok Lee10, Walter Kukull11, Collin Liu12, Yian Gu9, Gyungah R. Jun13, Van Park3, Li-San Wang4, Tiffany Chow4
1Neurology Division, Department of Medicine, University of British Columbia, 2Memory and Aging Center, 3School of Nursing, Department of Community Health Systems, University of California San Francisco, 4Department of Pathology and Laboratory Medicine, University of Pennsylvania, 5Department of Pharmacology and Toxicology, University of Toronto, 6Department of Neurosciences, University of California San Diego, 7Departments of Epidemiology & Population Health of Neurology & Neurological Sciences, Stanford University, 8Department of Neurology, Rutgers New Jersey Medical School, Englewood Health, 9Taub Institute, Columbia University, 10Meyers College of Nursing, New York University, 11Department of Epidemiology, University of Washington, 12Department of Neurology, Keck School of Medicine, University of Southern California, 13Biomedical Genetics Section, Department of Medicine, Boston University
Objective:
This study assesses the inter-rater reliability of consensus diagnoses for Alzheimer’s disease (AD) across multiple sites in the Asian Cohort for Alzheimer’s Disease (ACAD) study
Background:

The ACAD study is a multi-site collaborative investigation aimed at exploring the genetic and environmental risk factors for AD among Asian Americans and Asian Canadians. With the rising prevalence of AD in these populations, ACAD aims to enhance our understanding of how these factors contribute to the disease's progression. Data collection was designed to minimize participant burden while considering the historical stigma of dementia in Asian cultures. Standardizing diagnostic procedures across sites ensures consistent, high-quality data.

Design/Methods:

We analyzed 42 participants who had previously undergone a consensus-based diagnostic process across seven ACAD recruiting sites. The cohort included a spectrum of cognitive abilities, from cognitively healthy to cognitively impaired. Cognitive assessments were conducted in six languages to accommodate linguistic diversity and preferences among participants. Two independent neurologists re-evaluated the consensus diagnoses. Cohen’s Kappa coefficient measured inter-rater reliability, providing a robust assessment of diagnostic consistency across evaluators and sites.

Results:

Our results showed an observed agreement rate of 88% between the consensus process at the recruiting sites and the two external raters. Cohen’s Kappa value was also calculated at 0.835, indicating a near-perfect agreement. This high level of consistency affirms the reliability of the ACAD study’s diagnostic protocol, demonstrating that standardized procedures across sites, even with language variability, yield consistent diagnostic outcomes.

Conclusions:

The high inter-rater agreement underscores the effectiveness of the ACAD consensus process in delivering reliable diagnoses. These findings emphasize the importance of maintaining standardized diagnostic protocols across multiple research sites. This study's success not only paves a way for large-scale research into Alzheimer's disease in underrepresented groups but also significantly advances global efforts to foster inclusivity in AD research.

10.1212/WNL.0000000000208695
Disclaimer: Abstracts were not reviewed by Neurology® and do not reflect the views of Neurology® editors or staff.