Javan Rasokat and Andra Lezza -- When Chatbots Go Rogue - Lessons Learned from Building and Defending LLM Applications

What happens when teams add large language models to real applications and discover that familiar AppSec controls are no longer enough? Andra Lezza and Javan Rasokat share lessons from building, breaking, and defending LLM-enabled systems at Sage and presenting their findings at DEF CON. They compare prompt injection with SQL injection, explain AI red teaming, and unpack hallucinations, retrieval-augmented generation, grounding, and model safeguards. The conversation also examines corporate data leaking through prompts, the limits of trusting model providers, and how the OWASP Top 10 for LLM Applications complements issues observed in production. Andra and Javan close with practical advice for developers, data scientists, and security teams: treat AI systems as a new attack surface, establish clear data boundaries, and test controls against realistic abuse cases.The Application Security Podcast is brought to you by Security Journey.About Security JourneySecurity Journey provides application security education for developers and everyone in the software development lifecycle.→ Learn more about Security JourneyConnect with Andra Lezza and Javan Rasokat:→ Andra Lezza on LinkedIn→ Javan Rasokat on LinkedIn→ Javan Rasokat→ AppSec VillageMentioned in this episode:→ Adversarial Misuse of Generative AI (Javan's blog article)→ TLDR newsletter→ The Cuckoo's Egg by Cliff Stoll→ AppSec Village→ DEF CON→ ChatGPT→ DeepSeek→ NIST AI Risk Management Framework→ OWASP Top Ten for LLM Applications project homepageFollow the Application Security Podcast:➜ Home➜ X➜ LinkedIn➜ YouTube➜ Instagram➜ FacebookChapters:00:00 When Chatbots Go Rogue with Andra Lezza and Javan Rasokat01:49 Andra’s path into application security02:56 Javan’s path into application security04:38 Lessons from building and defending LLM applications08:22 Prompt injection compared with SQL injection11:46 Critical vulnerabilities found in real AI systems12:19 What AI red teaming actually means13:46 Hallucinations, RAG, and grounding19:51 Model safeguards and harmful requests20:35 Common AI development and deployment mistakes23:20 Corporate data exposure through prompts29:59 OWASP Top 10 for LLMs versus real-world findings32:02 Practical security advice for AI developers34:36 Bringing security practices to data scientists45:37 Key takeaways for defending LLM applications