Reliability of Unsupervised Digital Cognitive Assessment in a Large Adult Population Across the Aging Lifespan from INTUITION: A Brain Health Study
Michael G. Erkkinen1, Monroe Butler2, Roland Brown2, Matthew Hobbs2, Audrey Gabelle2, Andrew Becker2, Paramita Saha-Chaudhuri2, Jessica Ash2, Sean Kenny2, Hanson Lenyoun3, Matt Bianchi3, Shibeshih Belachew2
1Brigham & Women’s Hospital, Harvard Medical School, MA USA; Intuition Study Scientific Committee, 2Biogen Digital Health, Biogen Inc., Cambridge, MA USA, 3Apple Inc., Cupertino, CA USA
Objective:
To assess the feasibility and reliability of an unsupervised, digital cognitive battery performed by participants enrolled in the INTUITION (NCT05058950) study, who represent a U.S. adult population at-risk for cognitive decline.
Background:
Mobile and remote cognitive assessments may expedite patient screening for populations vulnerable to accelerated brain aging. As access to personal electronics increases, the modalities for data collection widen. Emerging evidence supports the case for the feasibility and validity of remote, digital cognitive assessments. However, test-retest reliability remains unknown in unsupervised settings and when performed at scale in diverse, heterogenous populations.
Design/Methods:
The INTUITION study is an observational two-year virtual study of 23,000 adults across the aging lifespan. Participants complete a monthly web-based 30-minute digital cognitive battery on a personal computing device. The assessment contains five tasks from the Cambridge Neuropsychological Test Automated Battery (CANTAB), which is designed to interrogate performance on specific cognitive domains, including attention, executive function, and episodic memory. We assessed test-retest reliability for key cognitive outcome variables with Pearson and intra-class correlations.
Results:
The INTUITION study was 91% enrolled (N=20,916) as of September 2022. N=9,415 participants have provided at least 5 sequential CANTAB assessments. Moderate-to-high test-retest reliability was observed for key outcome variables (range; r=0.42-0.80). Subtle, but measurable learning effects were noted across initial sessions and then abated alongside rising performance reliability following subsequent assessments. We present detailed test-retest reliabilities for representative outcomes of each CANTAB assessment from the total study population and by cohort.
Conclusions:
We demonstrated moderate-to-strong reliability in key cognitive outcome measures and reliability increased as a function of age. Reliability findings were comparable to those reported in the literature for supervised cognitive assessments. These initial results suggest remote and unsupervised cognitive testing is plausible and sufficiently reliable and can be self-administered in ecologically valid real-world settings.