Bridging the Knowledge Gap: Real-world Implementation of an AI Clinical Assistant in Neurology
Mikael Guzman Karlsson1, Douglas Wells2, Lauren Hess3, Thomas Chong1
1Pathology, 2Pediatric Neurology and Developmental Neuroscience, 3Pediatrics, Baylor College of Medicine | Texas Children's Hospital
Objective:
To implement and evaluate an AI-enabled clinical assistant utilizing retrieval-augmented generation and agentic capabilities for clinical decision support in neurology.
Background:

Neurologists require immediate access to complex protocols and dosing guidance during time-sensitive decisions. Traditional resources are fragmented across platforms and difficult to access at point of care. Knowledge gaps affect care quality, patient safety, and efficiency. An AI assistant providing source-cited responses from vetted institutional knowledge with integrated task support could address these barriers.

Design/Methods:

We deployed Brainy McSynapse within a pediatric neurology division using Microsoft Copilot Studio with GPT-4.1, disabling web and open-domain knowledge to restrict responses exclusively to curated institutional content (pediatric neurology handbook, algorithms, professional guidelines) with source citations. Development followed design-based research with semi-structured interviews identifying priority domains. An interdisciplinary advisory committee including informaticians, educators, ethicists, nurses, and pharmacists ensured oversight. Features included conversational decision trees for complex protocols, autonomous notifications via RSS feeds and calendar triggers, and task-oriented agentic capabilities. Parallel AI literacy curriculum supported appropriate use. Analytics tracked adoption, time-of-use, topic distribution, repeat engagement, re-query patterns, citation click-through, response latency, and source attribution rate.

Results:

Thirty-seven unique users generated 414 queries with sustained 25% week-over-week growth; 74% occurred after-hours. User satisfaction reached 75%; system errors were rare (2%). Most unsatisfactory responses reflected knowledge base gaps rather than technical failures, informing targeted content expansion. Query analysis revealed predominant themes of medication dosing and protocolized care. Top topics included status epilepticus management, antiseizure medication dosing, epilepsy pathways, and brain death protocols. Secondary analytics demonstrated high repeat engagement, active citation exploration, and re-query clustering around dosing edge cases.

Conclusions:

Restricted-source, retrieval-augmented AI with agentic capabilities achieved rapid adoption and high satisfaction in specialized neurology practice. Query pattern analysis reveals precise educational needs and knowledge base priorities. This framework demonstrates feasibility of knowledge-restricted AI enhancing clinical decision-making while maintaining source verification.

10.1212/WNL.0000000000216457
Disclaimer: Abstracts were not reviewed by Neurology® and do not reflect the views of Neurology® editors or staff.