Ep. 59: Alex Shan, Judgment Labs CEO

Alex Shan is the CEO of Judgment Labs (judgmentlabs.ai), where he's working on building agent behavior monitoring infrastructure. Before Judgment, he worked at Juniper Networks and Stanford AI Lab.Delta Institute (deltainstitutes.org) supports exceptional researchers and engineers, from academia to industry and beyond. They host technical events to bring great people together, a podcast that gives industry/academic leaders a platform to share their experiences, a small fellows program that builds a tight-knit community of exceptional people, and a grant program that provides compute/mentorship for research projects.Timestamps:00:00 Mission and Evals Focus00:30 Founder Background02:55 Childhood Co-Founders04:49 Stanford to Industry Pivot07:32 Juniper Agents Experience08:55 Founding Judgment Labs11:14 Why Existing Tools Fail13:23 Deep Agent Observability Model15:56 JudgeEval Open Core Strategy18:56 Evals Advice and Pitfalls23:24 Production Grounded Evals24:12 Rubric Discovery Signals25:06 Benchmarks That Evolve26:24 Legal Redlines Case Study27:22 From Edits To Rubrics30:40 Monitoring First Strategy32:09 Self Improving Agent Loop34:12 Competitive Differentiation36:13 Deep Context Evals42:43 Future Data Intelligence45:19 Closing Thoughts