LLM Observability: Lessons From MLOps w/ Maria Vechtomova (Cauchy)

For nine years, Maria Vechtomova was shouting about monitoring. Nobody cared, until LLMs arrived. As co-founder of Cauchy, Databricks MVP, and one of the most followed voices in MLOps, Maria has watched the field evolve from hand-built experiment trackers to today's flood of observability tools, and her central claim might surprise you: globally, nothing has changed. The fundamentals are the same: track your code, data, and models so you can roll back when something breaks. What did change is the surface area. Tools, prompts, embeddings, agents, every component shifts behavior unpredictably, and business metrics often become the only signal left. Maria gets into why most teams still can't roll back cleanly, why 40% of ML projects ship with no monitoring at all, and why she believes the next era of MLOps will be its biggest yet.