EPISODE · Jul 21, 2026 · 28 MIN
Is word error rate just a vanity metric?
from Deep Learning with PolyAI · host Team PolyAI
Send us Fan MailVoice AI vendors love to quote word error rate, usually somewhere around 2 to 3%, as proof their system understands customers. Oliver Shoulson, Agent Design & Engineering Lead at PolyAI, thinks it's one of the most misleading numbers in the business. He and Nikola Mrkšić go through a month of analysis across more than 100,000 real production call turns to explain why. Turns out not all words carry the same weight, most transcription errors are harmless, and task success barely moves even when the system mishears, because today's models recover the way a person would. Listen to the full episode, and see how PolyAI builds dialog agents that get this right at https://poly.ai?utm_source=youtube&utm_medium=podcast&utm_campaign=podcast&utm_content=podcastFollow PolyAI on LinkedInWatch this and other episodes of the Deep Learning pod on YouTube
Embed this episode
What this episode covers
Send us Fan Mail Voice AI vendors love to quote word error rate, usually somewhere around 2 to 3%, as proof their system understands customers. Oliver Shoulson, Agent Design & Engineering Lead at PolyAI, thinks it's one of the most misleading numbers in the business. He and Nikola Mrkšić go through a month of analysis across more than 100,000 real production call turns to explain why. Turns out not all words carry the same weight, most transcription errors are harmless, and task suc...
NOW PLAYING
Is word error rate just a vanity metric?
No transcript for this episode yet
Similar Episodes
No similar episodes found.
Similar Podcasts
No similar podcasts found.