Metacognition Against Confident Hallucinations

This episode explores a 2026 paper arguing that the real trust problem in language models is not error alone, but confident error, and that improving trust may depend more on metacognition than on simply scaling up knowledge. It unpacks key distinctions such as knowledge boundaries, calibration, discrimination, and the gap between intrinsic uncertainty and the uncertainty a model expresses in words, using factoid question answering as a clean test bed where correctness is measurable. The discussion also situates the paper within prior work on self-knowledge, verbalized uncertainty, and self-correction, while stressing that many apparent factuality gains may come from expanded knowledge or external tools rather than genuine awareness of limits. A listener would find it interesting because it reframes hallucinations as a trust and decision-making problem, and offers a sharper way to judge whether AI systems actually know when they should hedge, abstain, or seek evidence. Sources: 1. Hallucinations Undermine Trust; Metacognition is a Way Forward — Gal Yona, Mor Geva, Yossi Matias, 2026 http://arxiv.org/abs/2605.01428 2. Language Models (Mostly) Know What They Know — Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Ethan Perez, Deep Ganguli, Dario Amodei, Jack Clark, Jared Kaplan and collaborators, 2022 https://scholar.google.com/scholar?q=Language+Models+%28Mostly%29+Know+What+They+Know 3. Teaching Models to Express Their Uncertainty in Words — Stephanie Lin, Jacob Hilton, Owain Evans, 2022 https://scholar.google.com/scholar?q=Teaching+Models+to+Express+Their+Uncertainty+in+Words 4. What Large Language Models Know and What People Think They Know — Mark Steyvers, Heliodoro Tejeda, Aakriti Kumar, Catarina Belem, Sheer Karny, Xinyue Hu, Lukas W. Mayer, Padhraic Smyth, 2025 https://scholar.google.com/scholar?q=What+Large+Language+Models+Know+and+What+People+Think+They+Know 5. When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs — Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, Rui Zhang, 2024 https://scholar.google.com/scholar?q=When+Can+LLMs+Actually+Correct+Their+Own+Mistakes%3F+A+Critical+Survey+of+Self-Correction+of+LLMs 6. Can LLMs Express Their Uncertainty in Their Generated Responses? — Gal Yona, Roee Aharoni, Mor Geva, Yossi Matias, 2024 https://scholar.google.com/scholar?q=Can+LLMs+Express+Their+Uncertainty+in+Their+Generated+Responses%3F 7. Faithful or Fluent? Evaluating Natural Language Explanations of Uncertainty — Ghafouri et al., 2024 https://scholar.google.com/scholar?q=Faithful+or+Fluent%3F+Evaluating+Natural+Language+Explanations+of+Uncertainty 8. TruthfulQA: Measuring How Models Mimic Human Falsehoods — Stephanie Lin, Jacob Hilton, Owain Evans, 2022 https://scholar.google.com/scholar?q=TruthfulQA%3A+Measuring+How+Models+Mimic+Human+Falsehoods 9. Survey of Hallucination in Natural Language Generation — Ziwei Ji, et al., 2023 https://scholar.google.com/scholar?q=Survey+of+Hallucination+in+Natural+Language+Generation 10. The Geometry of Truth: Emergent Linear Structure in LLM Representations of Factuality — Marks and Tegmark, 2023 https://scholar.google.com/scholar?q=The+Geometry+of+Truth%3A+Emergent+Linear+Structure+in+LLM+Representations+of+Factuality 11. The Internal State of an LLM Knows When It's Lying — Levinstein and Herrmann, 2023 https://scholar.google.com/scholar?q=The+Internal+State+of+an+LLM+Knows+When+It%27s+Lying 12. Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations — Ziwei Ji, Lei Yu, Yeskendir Koishekenov, Yejin Bang, Anthony Hartshorn, Alan Schelten, Cheng Zhang, Pascale Fung, Nicola Cancedda, 2025 https://scholar.google.com/scholar?q=Calibrating+Verbal+Uncertainty+as+a+Linear+Feature+to+Reduce+Hallucinations 13. Calibrating the Voice of Doubt: How LLMs Diverge from Humans in Verbal Uncertainty — Jinhao Duan, Zicheng Liu, Zijie Liu, Kaidi Xu, Tianlong Chen, 2025 https://scholar.google.com/scholar?q=Calibrating+the+Voice+of+Doubt%3A+How+LLMs+Diverge+from+Humans+in+Verbal+Uncertainty 14. More Is Not Better: Visual Uncertainty Cues and the Fragility of Trust Calibration in LLM-Assisted Decision Making — authors not recovered from snippet, 2026 https://scholar.google.com/scholar?q=More+Is+Not+Better%3A+Visual+Uncertainty+Cues+and+the+Fragility+of+Trust+Calibration+in+LLM-Assisted+Decision+Making 15. Do Large Language Models Know What They Don't Know? — Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, Xuanjing Huang, 2023 https://scholar.google.com/scholar?q=Do+Large+Language+Models+Know+What+They+Don%27t+Know%3F 16. KnowRL: Teaching Language Models to Know What They Know — Sahil Kale, Devendra Singh Dhami, 2025 https://scholar.google.com/scholar?q=KnowRL%3A+Teaching+Language+Models+to+Know+What+They+Know 17. What Models Know, How Well They Know It: Knowledge-Weighted Fine-Tuning for Learning When to Say "I Don't Know" — Joosung Lee, Hwiyeol Jo, Donghyeon Ko, Kyubyung Chae, Cheonbok Park, Jeonghoon Kim, 2026 https://scholar.google.com/scholar?q=What+Models+Know%2C+How+Well+They+Know+It%3A+Knowledge-Weighted+Fine-Tuning+for+Learning+When+to+Say+%22I+Don%27t+Know%22 18. Selective-LAMA: Selective Prediction for Confidence-Aware Evaluation of Language Models — Hiyori Yoshikawa, Naoaki Okazaki, 2023 https://scholar.google.com/scholar?q=Selective-LAMA%3A+Selective+Prediction+for+Confidence-Aware+Evaluation+of+Language+Models 19. Selective Generation for Controllable Language Models — Minjae Lee, Kyungmin Kim, Taesoo Kim, Sangdon Park, 2024 https://scholar.google.com/scholar?q=Selective+Generation+for+Controllable+Language+Models 20. Dynamic Uncertainty Ranking: Enhancing Retrieval-Augmented In-Context Learning for Long-Tail Knowledge in LLMs — Shuyang Yu, Runxue Bao, Parminder Bhatia, Taha Kass-Hout, Jiayu Zhou, Cao Xiao, 2025 https://scholar.google.com/scholar?q=Dynamic+Uncertainty+Ranking%3A+Enhancing+Retrieval-Augmented+In-Context+Learning+for+Long-Tail+Knowledge+in+LLMs 21. UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation — Zixuan Li, Jing Xiong, Fanghua Ye, Chuanyang Zheng, Xun Wu, Jianqiao Lu, Zhongwei Wan, Xiaodan Liang, Chengming Li, Zhenan Sun, Lingpeng Kong, Ngai Wong, 2024 https://scholar.google.com/scholar?q=UncertaintyRAG%3A+Span-Level+Uncertainty+Enhanced+Long-Context+Modeling+for+Retrieval-Augmented+Generation 22. AI Post Transformers: Can LLMs Judge Their Own Capabilities? — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-05-01-can-llms-judge-their-own-capabilities-d78fed.mp3 23. AI Post Transformers: Do Language Models Know Their Limits — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-05-01-do-language-models-know-their-limits-48e444.mp3 24. AI Post Transformers: Teaching Language Models to Verbalize Uncertainty — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-05-01-teaching-language-models-to-verbalize-un-a1d774.mp3 25. AI Post Transformers: Experimental Comparison of Agentic and Enhanced RAG — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-04-14-experimental-comparison-of-agentic-and-e-37d8bc.mp3 Interactive Visualization: Metacognition Against Confident Hallucinations