Remote Assessment of Cognitive Frailty in Older Adults via Speech and Video Analysis
Ram Kinker Mishra1, Mohammad Dehghan Rouzi2, Adonay Nunes1, Myeounggon Lee2, Ashkan Vaziri1, Bijan Najafi2
1BioSensics, 2Baylor College of Medicine
Objective:
To develop and validate a video-based tool for assessment of cognitive frailty.
Background:
Cognitive frailty is a complex and multifaceted condition characterized by the coexistence of physical frailty and cognitive impairment and is prevalent in older adults. Traditional clinical assessments for frailty can be time-consuming and costly. There is a need for a robust solution that can be administered more frequently than traditional assessments.
Design/Methods:
Participants aged 20-85 performed 20-second elbow extension-flexion tasks under both single-task (ST) and dual-task (DT) conditions. During DT, participants counted backward while performing the task, video-recorded via webcam. Frailty-related measurements were derived from video-based motion tracking algorithms which detect and analyze hand movements with high fidelity. The speech features were analyzed using BioDigit Speech (BioSensics LLC, Newton, MA USA), which enables measurement of over 40 speech outcomes by leveraging information in the audio waveform, spectrum, and cepstrum to capture phonatory, articulatory, prosody, and intelligibility features of speech.
Results:
Eighty-two participants were categorized into four groups: Robust (CI+/PF+, n=28), High Cognition (CI+/PF-, n=17), High Physical (CI-/PF+, n=21), and Worst (CI-/PF-, n=16). A machine learning model, employing a random forest approach with 5-fold cross-validation, was developed with both video and speech features as the input features. The model achieved an accuracy and precision of more than 75% along with an area under the curve (AUC) value of more than 0.80. Notably, the contribution of video features collected during the dual-task (DT) condition was found to be more significant compared to the single-task (ST) condition.
Conclusions:
The study validated the capacity of the proposed solution to evaluate cognitive frailty remotely using the video and speech biomarkers. The development of a video-based assessment for cognitive frailty offers a cost-effective, remote, and scalable solution that can be administered more frequently than traditional clinical assessments.