AI-powered Translation In Neurology: Potential Gains and Ongoing Challenges For Use In The Clinic
Michelle You1, Chase Goldberg1, Michael Vazquez1, Dhruba Podder1, Rahim Hirani1, Mill Etienne2
1School of Medicine, New York Medical College, 2Department of Neurology, Westchester Medical Center
Objective:
To examine the contextual accuracy of artificial intelligence (AI), specifically Google Translate (GT), in scripted case-based encounters in Neurology.
Background:
The integration of AI in healthcare holds vast potential in its applications, particularly as a tool for medical interpretation.1 According to the U.S. Census, more than 25 million people have limited English proficiency (LEP).8 With studies linking language discordance to poorer health outcomes2,3,4, optimizing communication between physicians and their patients is critical. AI-powered, cost-free tools offer an accessible alternative to professional interpreter services.
Design/Methods:
Using GT, four scripts simulating Neurology encounters were translated into nine languages including Arabic, Bengali, Farsi, Haitian-Creole, Mandarin-Chinese, Pashto, Spanish, Urdu, and Yiddish. Study staff identifying as advanced or native speakers in each target language evaluated the translations using a validated scale for machine translation accuracy.5,6 Using a 5-point Likert scale (1 = lowest accuracy and 5 = best accuracy), physician and patient script translations were assessed separately by one evaluator in each language across four categories: Fluency, Adequacy, Meaning, and Severity.
Results:
A total score was calculated for each scale category by averaging ratings across all languages, with higher scores indicating better translation accuracy. Translations maintained Fluency (M=3.9), Adequacy (M=4.2), Meaning (M=4.1), and Severity (M=4.35) for physician scripts, with similar ratings for patient scripts in Fluency (M=3.9), Adequacy (M=4.3), Meaning (M=4.3), and Severity (M=4.43). Overall, translations preserved communication of content without overt evidence of patient harm. Evaluators noted the use of more formal language in GT translations, with occasional contextual inaccuracies due to punctuation or inexact equivalence.
Conclusions:
While the advent of AI in healthcare is poised to shape the efficiency and accessibility of care, its use brings complex ethical and technical questions. Understanding the clinical utility and efficacy of AI, especially in medical interpretation, is of vital importance to ensure its safe and responsible use.
Disclaimer: Abstracts were not reviewed by Neurology® and do not reflect the views of Neurology® editors or staff.