How to Evaluate AI You Can Trust E166

How do you know if an AI system is trustworthy, compliant, ethical, and fit for purpose?  In this episode of the FIT4Privacy Podcast, Punit Bhatia is joined by Stella Liu, an AI evaluation expert and founder of AI Evals & Analytics, to unpack one of the most practical and overlooked challenges in AI today: how to evaluate AI systems before and after deployment.  KEY MOMENTS 02:09 —AI Definition 03:02 —AI Evaluations 10:31 — Why AI Testing Is Hard 14:06 — Evals Plus Analytics 18:15 —Synthetic Data 23:47 —Protecting Privacy Ethical 29:05 — AI Evals as a Company 29:52 —How to reach Stella Liu Stella explains why AI behaves differently from traditional software, why testing code alone is no longer enough, and how AI evaluations (AI evals) help organizations assess real-world behavior, risk, and performance. From evaluation driven development to continuous monitoring in production, the conversation explores how teams can move beyond guesswork and hype toward repeatable, measurable AI governance.  ⸻ ABOUT THE GUEST Stella Liu is the Co-founder of AI Evals & Analytics (Maven), where she created the AI Evals & Analytics Playbook and teaches top-rated courses on LLM evaluation, monitoring, and product alignment. She’s also the Head of AI Applied Science at ASU, leading evals and analytics across university-wide AI products and building higher-ed’s first formal AI evaluation framework, and she previously led data science at Shopify and Carvana with 12+ years shipping large-scale ML systems.  ABOUT THE HOST    Punit Bhatia is one of the leading privacy experts who works independently and has worked with professionals in over 30 countries. Punit works with business and privacy leaders to create an organization culture with high privacy awareness and compliance as a business priority. Selectively, Punit is open to mentor and coach privacy professionals.   ⸻ Resources & Links  Guest Links Stella Lui • Website: https://maven.com/  • LinkedIn: https://www.linkedin.com/in/wenxingl/  Grow Skills (Privacy Courses & Insights) • Courses: https://growskills.store/courses/ • Insights: https://growskills.store/insights/ • Website: https://growskills.store/  FIT4Privacy • Website: https://www.fit4privacy.com • Podcast: https://www.fit4privacy.com/podcast • Blog: https://www.fit4privacy.com/blog • YouTube: http://youtube.com/fit4privacy  Punit Bhatia • Website: https://www.punitbhatia.com  Books  • Be Ready for GDPR • AI & Privacy – How to Find Balance • Intro to GDPR • Be an Effective DPO