This Week in AI Security - 6th November 2025

In this week's episode, Jeremy looks at three compelling stories and a significant academic paper that illustrate the accelerating convergence of AI, APIs, and network security.API Exposure in AI Services: We discuss a path traversal vulnerability that led to the discovery of 3,000 API keys in a managed AI hosting service, underscoring that the API remains the exposed attack surface where data exfiltration occurs.AI Code Agent Traffic Analysis: Drawing on research from Chaser Systems, Jeremy breaks down the network traffic from popular AI coding agents (like Copilot and Cursor). The analysis reveals that sensitive data, including previous conversation context and PII, is repeatedly packaged and resent with every subsequent request, making detection and leakage risk significantly higher.LLM-Powered Malware: We cover a groundbreaking discovery by the Microsoft Incident Response Team (DART): malware using the OpenAI Assistants API as its Command and Control (C2) server. This new category of malware replaces traditional hard-coded instructions with an LLM-driven "brain," giving it the potential to coordinate malicious activity with context, creativity, and adaptability.The Guardrail Fallacy: Finally, Jeremy discusses an academic paper showing that strong, adaptive attacks can bypass LLM defenses against Jailbreaks and Prompt Injections with an Attack Success Rate (ASR) of over 90%. The research argues that simple guardrails provide organizations with a dangerous false sense of security.Episode Linkshttps://chasersystems.com/blog/what-data-do-coding-agents-send-and-where-to/https://embracethered.com/blog/posts/2025/claude-abusing-network-access-and-anthropic-api-for-data-exfiltration/ https://arxiv.org/pdf/2510.09023https://www.microsoft.com/en-us/security/blog/2025/11/03/sesameop-novel-backdoor-uses-openai-assistants-api-for-command-and-control/------Worried about AI security? Get Complete AI Visibility in 15 Minutes. Discover all of your shadow AI now. Book a demo of Firetail's AI Security & Governance Platform: https://www.firetail.ai/request-a-demo