Objective:

To determine the reliability of artificial intelligence (AI) as a tool to obtain information about the effect of gender-affirming hormone therapy (GAHT) on neurological health.

Background:

The recent surge in access to AI platforms has the potential to transform how laypeople obtain health information, particularly among historically disadvantaged populations hesitant to seek medical care. However, these platforms have not consistently proven to be reliable sources of accurate information and so may cause harm to patients. The transgender community, known for its high rates of reluctance to pursue medical care, may increasingly turn to online resources, thereby putting individuals at risk of encountering misinformation. Given the well-documented health disparities affecting this population, preventing the exacerbation of these disparities is an increasingly critical challenge. We sought to evaluate the ability of commonly used AI platforms to provide accurate information about neurological health for transgender patients.

Design/Methods:

We conducted a stress test of four different AI platforms (GPT-4o, Claude 3.5 Sonnet, Google Gemini, and Meta AI) to evaluate the quality of their responses to questions about the impact of GAHT on risk of stroke and seizure. The responses were compared to guidelines from the World Professional Association for Transgender Health (WPATH) Standards of Care Version 8 (SOC8). When there were no official WPATH guidelines, we used the best available evidence from peer-reviewed scientific papers.

Results:

Most responses reflected the uncertainty and lack of consensus in the field. All platforms recommended seeking personalized guidance from a healthcare provider. However, the length of each platform’s responses varied significantly, and academic citations were not consistently provided by any platform. 

Conclusions:

Our work demonstrates a need for higher standards of depth and utilization of evidence for AI platforms to become reliable tools for patients and clinicians alike. This also demonstrates the need for more research on the impact of gender-affirming care.