Voice AI - The Next Frontier | Decagon, Retell, Vapi, Smallest AI, Daily

Voice agents are one of the hottest use cases in enterprise right now, but also one of the hardest to actually take live. Getting latency low enough to feel human without dumbing down the responses, making reliable tool calls to a CRM without dropping the customer mid-call, building fallback models for when Anthropic or OpenAI are running hot. None of it is as simple as the demos make it look.Last week I hosted a fireside chat with five eng leaders who deal with this stuff every day: Basia Sudol (Head of Enterprise Solutions, Decagon), Varun Singh (CPTO, Daily), Steven Diaz (FDE Manager, Vapi), Tyler D'Silva (Founding FDE, Retell AI), and Sudarshan Kamath (Founder, Smallest AI).We get into why nobody serious is shipping real-time voice-to-voice yet, why LLMs forget the middle of your prompt (and what that does to your architecture), why a giant prompt quietly destroys your unit economics, and why voice agent costs are now being compared directly against human labor.Plus the stuff nobody warns you about: turn-taking, HIPAA constraints, why outbound is easier than inbound, why getting an exec to actually like the voice can be harder than any model problem, and more!Chapters below:00:00 Intro00:26 Meet the panel01:23 Daily, WebRTC, and 20 years of building voice03:49 How Smallest AI made real-time TTS work05:23 Why Decagon moved into voice08:16 How Vapi and Retell think about the stack11:14 Forward deployed vs solutions engineering16:48 Voice agent architecture, explained simply22:11 Cascade vs speech-to-speech: the real tradeoff28:11 Hybrid pipelines and mixing models32:16 Accents, multilingual, and getting Singlish right35:32 Prompts vs workflows, and the latency fight44:26 How you actually evaluate a voice agent49:04 Simulation-based evals49:49 What production metrics really look like51:53 Building a QA framework that scales54:53 Evaluating speech-to-speech58:10 Open source benchmarks59:57 Why picking a voice is so subjective01:02:03 Personalization and custom voices01:03:20 Voice quality is solved, GPU efficiency is the new war01:06:23 Why outbound calls work better than you'd think01:10:38 Deploying in regulated industries (HIPAA, retention, audits)01:12:43 Turn-taking, the hardest unsolved problem in voice01:18:53 Where voice agents go in the next year01:27:55 Audience Q&A: inside Smallest's Hydra model01:32:15 The deployment problems nobody has solved yet01:37:46 Closing thoughts and thanks