How to Solve AI's Biggest Problem | Atin Sanyal, Galileo

How do you know whether an AI agent is doing its job or quietly failing in production?Galileo is building the trust layer for AI. Its evaluation and observability platform is how enterprises measure whether the output of an LLM or an agent is good or bad.Galileo started before "LLM" was even a word. When Atin showed his prototype to Stanford's Chris Ré, his own first question was "what is a language model?" Today its customers include Reddit, Airbnb, P&G, Comcast, and six of the Fortune 50. Atin spent a decade in big tech before co-founding Galileo with Vikram Chatterji in early 2021. He worked on the knowledge graphs behind Siri at Apple, then became one of the leads and architects of Michelangelo, Uber's AI platform, that hosts thousands of models across pricing, ETA, and demand.That Uber experience taught him the lesson the whole company is built on; that in AI, observability and evaluation are the real bottleneck, and bad data is catastrophic.As ChatGPT turned every AI output into something a user sees directly, the measurement problem went from academic to mission-critical. So Atin made a contrarian bet: instead of using giant LLMs to judge other LLMs, Galileo built Luna, small 1-3B parameter models that run evals at breakthrough latencies of 100 milliseconds and below.If you are excited about how AI actually gets shipped, trusted, and controlled inside real enterprises, this episode is for you.00:00 - Trailer01:14 - From India to Apple, Uber, and Galileo01:34 - Where the name "Galileo" came from02:38 - Building Siri's early knowledge graphs at Apple03:29 - Becoming an architect of Uber's Michelangelo05:15 - Why every AI output is now mission-critical06:45 - How Atin and Vikram zeroed in on Galileo07:42 - "What is a language model?"09:38 - Building the world's first feature store at Uber11:27 - Language models and tokens, explained simply14:19 - Where the observability insight came from15:53 - Quantifying uncertainty and hallucinations16:36 - The first customers and first use case19:15 - How the product evolved from a data scientist tool23:18 - Why ChatGPT changed everything for Galileo23:57 - The enterprise AI adoption curve, 2021 to 202626:35 - Why they built the Luna model28:32 - Turning LLM "writers" into "calculators"28:51 - Attacking the latency problem31:48 - Luna: the modeling and infrastructure innovation33:09 - What evals are, and why they blew up34:26 - The case for small language models36:58 - What "general reasoning" really means40:39 - AI usage is exploding — and why that matters43:08 - Online vs offline: the "it worked on my machine" problem44:33 - The evals flywheel and evals-driven development46:56 - Galileo in a nutshell47:39 - What real agents in production look like today49:30 - A sales intelligence platform, powered by Galileo50:47 - The agent control product52:12 - Building GTM as a hardcore engineer from India54:43 - Garbage in, garbage out: nailing the ICP55:42 - How the pitch changed from customer 1 to 2057:27 - Why Atin switched from CTO to CPO-------------India’s talent has built the world’s tech—now it’s time to lead it.This mission goes beyond startups. It’s about shifting the center of gravity in global tech to include the brilliance rising from India.What is Neon Fund?We invest in seed and early-stage founders from India and the diaspora building world-class Enterprise AI companies. We bring capital, conviction, and a community that’s done it before.Subscribe for real founder stories, investor perspectives, economist breakdowns, and a behind-the-scenes look at how we’re doing it all at Neon.-------------Check us out on:Website: https://neon.fund/Instagram: https://www.instagram.com/theneonshoww/LinkedIn: https://www.linkedin.com/company/beneon/Twitter: https://x.com/TheNeonShowwConnect with Siddhartha on:LinkedIn: https://www.linkedin.com/in/siddharthaahluwalia/Twitter: https://x.com/siddharthaa7-------------This video is for informational purposes only. The views expressed are those of the individuals quoted and do not constitute professional advice.Send us Fan Mail