EPISODE · Jul 15, 2026 · 32 MIN
EP 45: Why AI Agents Break in Production: The Missing Harness in Your Data Stack
from Data Science With Sam · host Soumava Dey
An independent evaluation of Snowflake's Cortex Analyst found 6 in 10 AI-generated queries were wrong — but they all compiled and ran without a single error. That's not a model problem. That's a missing harness problem. Pradnesh Patil is the Co-Founder and CEO of Altimate AI — a platform bringing agentic AI to data engineering with tools trusted by Fortune 500s and downloaded more than 1 million times across 200+ countries. Before Altimate, he spent a decade in product leadership at Palo Alto Networks, Cisco, and VMware. IN THIS EPISODE: ▪ The five components of an agentic data engineering harness: context, governance, MCP tools, shared skills, and agent infrastructure — and why missing any one of them causes silent failures ▪ Why a system prompt cannot substitute for a harness: a prompt tells the model what to do, a harness tells it what is actually true ▪ Where the 27–33% phantom table references and 78% silent wrong joins come from — and why it's not the LLM's fault ▪ How Altimate Code topped ADE-Bench using Sonnet while competitors used Opus — proof that the harness matters more than the model ▪ The deterministic vs LLM boundary: why validation, cost checks, and query correctness are deterministic jobs and should never go to an LLM ▪ Context compaction innovation: why standard LLM compaction destroys long-running data engineering tasks — and how Altimate fixed it ▪ The $5,000 Cortex AI query bill — and how permission-based governance controls prevent agents from going rogue on your cloud bill ▪ The future of data engineering: days of writing SQL by hand are ending — what the data engineer's role becomes in an agentic world ▪ Pradnesh's advice: build open source, build cross-platform — avoid siloed AI features that don't move the industry forward FIND PRADNESH: Website https://altimate.ai/ Github for altimate-code: https://github.com/AltimateAI/altimate-code Connect with Pradnesh on LinkedIn: https://www.linkedin.com/in/pradneshpatil/ DATASCIENCEWITHSAM: Weekly conversations with practitioners and builders at the frontier of AI, data science, and machine learning. Subscribe on Apple Podcasts, Spotify, Amazon Music, iHeartRadio, Podbean, and YouTube. If you enjoyed this episode, share it with a data engineer who is still wondering why their AI agent keeps writing queries for tables that don't exist.
What this episode covers
An independent evaluation of Snowflake's Cortex Analyst found 6 in 10 AI-generated queries were wrong — but they all compiled and ran without a single error. That's not a model problem. That's a missing harness problem. Pradnesh Patil is the Co-Founder and CEO of Altimate AI — a platform bringing agentic AI to data engineering with tools trusted by Fortune 500s and downloaded more than 1 million times across 200+ countries. Before Altimate, he spent a decade in product leadership at Palo Alto Networks, Cisco, and VMware. IN THIS EPISODE: ▪ The five components of an agentic data engineering harness: context, governance, MCP tools, shared skills, and agent infrastructure — and why missing any one of them causes silent failures ▪ Why a system prompt cannot substitute for a harness: a prompt tells the model what to do, a harness tells it what is actually true ▪ Where the 27–33% phantom table references and 78% silent wrong joins come from — and why it's not the LLM's fault ▪ How Altimate Code topped ADE-Bench using Sonnet while competitors used Opus — proof that the harness matters more than the model ▪ The deterministic vs LLM boundary: why validation, cost checks, and query correctness are deterministic jobs and should never go to an LLM ▪ Context compaction innovation: why standard LLM compaction destroys long-running data engineering tasks — and how Altimate fixed it ▪ The $5,000 Cortex AI query bill — and how permission-based governance controls prevent agents from going rogue on your cloud bill ▪ The future of data engineering: days of writing SQL by hand are ending — what the data engineer's role becomes in an agentic world ▪ Pradnesh's advice: build open source, build cross-platform — avoid siloed AI features that don't move the industry forward FIND PRADNESH: Website https://altimate.ai/ Github for altimate-code: https://github.com/AltimateAI/altimate-code Connect with Pradnesh on LinkedIn: https://www.linkedin.com/in/pradneshpatil/ DATASCIENCEWITHSAM: Weekly conversations with practitioners and builders at the frontier of AI, data science, and machine learning. Subscribe on Apple Podcasts, Spotify, Amazon Music, iHeartRadio, Podbean, and YouTube. If you enjoyed this episode, share it with a data engineer who is still wondering why their AI agent keeps writing queries for tables that don't exist.
NOW PLAYING
EP 45: Why AI Agents Break in Production: The Missing Harness in Your Data Stack
No transcript for this episode yet
Similar Episodes
Mar 26, 2026 ·1m
Jan 2, 2026 ·47m
Dec 21, 2025 ·46m