#28 - How AI Confidence Masks Medical Uncertainty

Can you trust a confident answer, especially when your health is on the line?This episode explores the uneasy relationship between language fluency and medical truth in the age of large language models (LLMs). New research asks these models to rate their own certainty, but the results reveal a troubling mismatch: high confidence doesn’t always mean high accuracy, and in some cases, the least reliable models sound the most sure.Drawing on her ER experience, Laura illustrates how real clinical care embraces uncertainty—listening, testing, adjusting. Meanwhile, Vasanth breaks down how LLMs generate their fluent responses by predicting the next word, and why their self-reported “confidence” is just more language, not actual evidence.We contrast AI use in medicine with more structured domains like programming, where feedback is immediate and unambiguous. In healthcare, missing data, patient preferences, and shifting guidelines mean there's rarely a single “right” answer. That’s why fluency can mislead, and why understanding what a model doesn’t know may matter just as much as what it claims.If you're navigating AI in healthcare, this episode will sharpen your eye for nuance and help you build stronger safeguards. Reference: Benchmarking the Confidence of Large Language Models in Answering Clinical Questions: Cross-Sectional Evaluation StudyMahmud Omar et al.JMIR (2025)Credits: Theme music: Nowhere Land, Kevin MacLeod (incompetech.com)Licensed under Creative Commons: By Attribution 4.0https://creativecommons.org/licenses/by/4.0/