Shipping Agents, Not Vulnerabilities with Ian Webster, PromptFoo CEO
from Alexa's Input (AI) Hosted by Alexa Griffith
As LLM apps evolve from simple chatbots to tool-using agents, the attack surface explodes, and the old security playbooks don’t hold. In this episode of Alexa’s Input (AI), Alexa Griffith sits down with Ian Webster, co-founder and CEO of PromptFoo, to break down what AI security actually looks like in practice: automated red teaming, prompt injection and jailbreak testing, evaluation workflows that scale, and why “guardrails alone” is not a security strategy. Ian shares how PromptFoo grew from a side project into a widely adopted open-source standard, what it means to raise multi-millions in a fast-moving market, and how enterprises are approaching the full vulnerability lifecycle, from finding issues to triage, remediation, and validation. Ian also discusses the “lethal trifecta” that makes agents fundamentally risky (untrusted input + sensitive data + exfil path), and why MCP security isn’t just about users and tools, it’s about dangerous tool combinations and rogue servers. Podcast Links Watch: https://www.youtube.com/@alexasinput Read: https://alexasinput.substack.com/ Listen: https://creators.spotify.com/pod/profile/alexagriffith/ More: https://linktr.ee/alexagriffith Website: https://alexagriffith.com/ LinkedIn: https://www.linkedin.com/in/alexa-griffith/ Find out more about the guest at: PromptFoo Website: https://www.promptfoo.dev/ Github: https://github.com/promptfoo/promptfoo Ian’s LinkedIn: https://www.linkedin.com/in/ianww/ Chapters 00:00 Introduction to AI Security Challenges 02:06 Funding and Growth of PromptFu 06:16 The Genesis of PromptFu 11:05 Career Journey and Lessons Learned 12:53 Understanding AI Red Teaming 17:36 Recent AI Security Vulnerabilities 19:46 The Dual Nature of AI in Security 21:47 Understanding the Lethal Trifecta in AI Security 24:22 Exploring Model Context Protocol (MCP) and Its Security Implications 26:22 Common Security Issues in MCP Systems 28:17 The Role of Identity and Permissions in AI Security 30:00 Practical Implications of Using PromptFoo for Developers 31:33 Evaluating Language Models: Challenges and Techniques 36:34 The Limitations of Guardrails in AI Security 38:25 Best Practices for Engineers in AI Development 39:58 Future Trends in AI and Security 42:28 Everyday Applications of AI and Language Models
Embed this episode
No transcript yet.