EPISODE · May 29, 2026 · 53 MIN
Principles of Evals: The Future of GenAI Evaluation (E.43)
from Free Form AI · host Michael Berk
LLMs are optimized to sound convincing—not to know when they’re wrong. In this episode, Deanna Emery breaks down why hallucinations are fundamentally tied to how language models work, why confidence is often disconnected from correctness, and how better evaluation strategies can make AI systems more reliable in production. We also get into uncertainty, semantic reasoning, and what humans still do better than models.00:00 — Why LLMs hallucinate confidently09:00 — The limits of current eval systems18:00 — Why uncertainty matters in AI27:00 — Semantic reasoning vs memorization38:00 — What humans still do better than modelsThe biggest risk in AI isn’t wrong answers. It’s wrong answers delivered with confidence.
What this episode covers
LLMs are optimized to sound convincing—not to know when they’re wrong. In this episode, Deanna Emery breaks down why hallucinations are fundamentally tied to how language models work, why confidence is often disconnected from correctness, and how better evaluation strategies can make AI systems more reliable in production. We also get into uncertainty, semantic reasoning, and what humans still do better than models.00:00 — Why LLMs hallucinate confidently09:00 — The limits of current eval systems18:00 — Why uncertainty matters in AI27:00 — Semantic reasoning vs memorization38:00 — What humans still do better than modelsThe biggest risk in AI isn’t wrong answers. It’s wrong answers delivered with confidence.
NOW PLAYING
Principles of Evals: The Future of GenAI Evaluation (E.43)
No transcript for this episode yet
Similar Episodes
No similar episodes found.
Similar Podcasts
No similar podcasts found.