Skip to content
AI Observability: Measuring Quality, Risk, and ROI episode artwork

Oct 13, 2026 25 min

AI Observability: Measuring Quality, Risk, and ROI

from The Tech Trek Hosted by Elevano

An AI agent can be fast, available, and still get the answer wrong. That changes what engineering teams need to measure. Jeetendra Soneja, VP of Engineering, AI Platform at iHerb, joins The Tech Trek to discuss why AI observability goes beyond traditional software monitoring. Teams need to understand not just latency and errors, but also response quality, groundedness, cost, and whether an agent completes its intended task. We also explore how to measure AI ROI without relying on misleading productivity metrics. Jeetendra shares his perspective on risk based guardrails, the skills engineers need, and why observability should begin during development rather than waiting until production. Key Takeaways Monitor agent quality and task completion alongside latency, availability, and cost. Build evaluations around the specific business problem the AI system is solving. Connect AI spending to business outcomes, not just token usage or code output. Start with basic observability, then expand controls as AI adoption grows. Episode Highlights 01:00 Why AI observability requires more than traditional monitoring. 06:00 The metrics that matter for cost, quality, and reliability. 08:00 How to measure AI ROI beyond productivity claims. 11:00 Setting risk tolerances and choosing appropriate guardrails. 16:00 The engineering skills becoming more valuable with AI. 22:00 Proving value before scaling AI systems. One Line That Stuck "It's one thing to get it done, it's another thing to get it done economically." Follow The Tech Trek for more conversations about building and operating modern technology teams.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

AI Observability: Measuring Quality, Risk, and ROI

0:00 25:11

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of The Tech Trek?

This episode is 25 minutes long.

When was this The Tech Trek episode published?

This episode was published on October 13, 2026.

Can I download this The Tech Trek episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.