When AI Knows the Diagnosis But Misses the Action

Mount Sinai just published the first independent safety evaluation of ChatGPT Health — and the findings should change how you think about AI in healthcare.Published in Nature Medicine, researchers ran 960 patient interactions across 21 medical specialties. What they found wasn't that ChatGPT gets medicine wrong. It's that it gets the diagnosis right, then tells you to do the wrong thing about it.In this episode, we break down:Why ChatGPT told patients to wait in over half of true emergencies — after correctly identifying the danger in its own explanationThe inverted suicide crisis alerts that fired for sadness but went silent when patients described specific plans for self-harmThe sycophancy problem: why ChatGPT is 12x more likely to agree when you downplay your own symptomsWhere ChatGPT actually performs well (93% in semi-urgent cases) — and why that makes the failures harder to spotWhat this means for anyone using, building, or recommending AI health toolsSources & Links:Primary Study — Nature Medicine, Feb 2026https://doi.org/10.1038/s41591-026-04297-7Mt. Sinai Press Releasehttps://www.mountsinai.org/about/newsroom/2026/research-identifies-blind-spots-in-ai-medical-triageForbes: "ChatGPT Provided Wrong Advice In Over 50% Medical Emergencies Tested"https://www.forbes.com/sites/brucelee/2026/03/08/chatgpt-provided-wrong-advice-in-over-50-medical-emergencies-tested/NPR: "ChatGPT might give you bad medical advice, studies warn"https://www.nhpr.org/2026-03-11/chatgpt-might-give-you-bad-medical-advice-studies-warnRelated: AI Chatbots and Medical Misinformation — Communications Medicine, 2025https://doi.org/10.1038/s43856-025-01021-3Full research brief and deep dive on Substack:danmccoymd.substack.com