![Ep 5 | Joshua Saxe - The Hard Part Isn't Building the Agent: Measuring Effectiveness | [un]prompted 2026 episode artwork](https://d3t3ozftmdmh3i.cloudfront.net/staging/podcast_uploaded_nologo/45690896/45690896-1775792431412-d94df1f86c828.jpg)
EPISODE · May 28, 2026 · 22 MIN
Ep 5 | Joshua Saxe - The Hard Part Isn't Building the Agent: Measuring Effectiveness | [un]prompted 2026
from [un]prompted Security Practitioner Con 2026 · host [un]prompted
Day 1 | Stage 1As AI coding tools drive the cost of building security agents toward zero, the hard problem becomes knowing whether they'll actually work against real attacks and vulnerabilities not seen before. Presents a practical journey from naive precision/recall metrics to multi-dimensional evaluation that captures reasoning quality, evidence gathering, and tool-calling logic. Shows how proper measurement unlocks automated agent improvement using genetic algorithms and AI coding tools. Includes live demo.Joshua Saxe is AI Security Technical Lead, Meta.Watch on YouTube: https://www.youtube.com/watch?v=rO2yA52U_i4
What this episode covers
Day 1 | Stage 1As AI coding tools drive the cost of building security agents toward zero, the hard problem becomes knowing whether they'll actually work against real attacks and vulnerabilities not seen before. Presents a practical journey from naive precision/recall metrics to multi-dimensional evaluation that captures reasoning quality, evidence gathering, and tool-calling logic. Shows how proper measurement unlocks automated agent improvement using genetic algorithms and AI coding tools. Includes live demo.Joshua Saxe is AI Security Technical Lead, Meta.Watch on YouTube: https://www.youtube.com/watch?v=rO2yA52U_i4
NOW PLAYING
Ep 5 | Joshua Saxe - The Hard Part Isn't Building the Agent: Measuring Effectiveness | [un]prompted 2026
No transcript for this episode yet
Similar Episodes
No similar episodes found.
Similar Podcasts
No similar podcasts found.