The Trojan Is Your Agent's Memory: Why Single-Step Defenses Miss Persistent Attacks

The Trojan Is Your Agent's Memory: Why Single-Step Defenses Miss Persistent Attacks Source: https://arxiv.org/abs/2605.31042 Paper was published on May 29, 2026 This episode was AI-generated on June 1, 2026. The script was written by an AI language model and the host voices were synthesized by Eleven Labs. The producer is not affiliated with Anthropic or Eleven Labs. The famous prompt-injection attack barely works against frontier models anymore — so why does a multi-step version succeed 95% of the time against the very same model? It's because the danger moved from the chat box into the agent's persistent memory, and a new paper argues the entire deployed safety industry is defending the wrong moment. The fix flips the question from 'is this action dangerous?' to 'where did this instruction come from?' Key Takeaways: - Why classic prompt injection now fails at near-zero, yet a slow attack smeared across files and sessions succeeds about 95% of the time against the same frontier model - The core reframe: the dangerous moment isn't the harmful action, it's the earlier innocent step when untrusted text quietly becomes a future instruction - How DASGuard's chain-of-custody provenance tracking — and its draft-vs-sent-email distinction between sanitizing files and blocking irreversible actions — cuts attack success from 95% to under 16% - The ablation that proves the insight is the contribution: remove just the source labels and the whole defense collapses back to 92.7%, even with detection and memory intact - Why the 16% number deserves grains of salt — no adaptive attacker, a benchmark and defense from the same team, a thin clean-task set, and a 13% false-positive rate - Why the reframe outlasts the benchmark: provenance tracking is portable across agent harnesses, but recovery from an already-poisoned workspace remains wide open 00:00 - The attack with no visible moment: An opening scenario where a planted policy line graduates into a trusted runbook rule and triggers harm days later, with no single step that looks dangerous. 02:54 - Why classic prompt injection stopped working: The authors run AgentDojo and InjecAgent against undefended frontier models and find single-shot injection now fails at near-zero — making the field think the problem is half-solved. 05:48 - The agentic harness and the persistence problem: How memory that survives across sessions creates a brand-new place for attackers to hide, and why the right question shifts from 'is this safe?' to 'where did this come from?' 08:43 - Relocating the trojan to the workspace: Borrowing the backdoor concept from classic security and pointing the trigger at persistent workspace state rather than a secret token or pixel pattern. 11:37 - ClawTrojan and the 95% number: How the benchmark builds runnable sandboxes and validates full multi-step attack chains — including fragmented payloads — that succeed roughly 95% of the time. 14:32 - How DASGuard works: detect, attribute, sanitize: A walkthrough of the three gates, the content-source graph that propagates suspicion across steps, and the shadow workspace that cleans files instead of just blocking. 17:26 - The results and the ablation that proves the point: DASGuard drops attack success to under 16% while nine baselines barely move the needle, and removing provenance alone reverts the defense to near-undefended. 20:21 - Where the numbers deserve skepticism: A steelman critique covering the same-team benchmark, the absence of an adaptive attacker, the thin clean-task set, false positives, and adapted baselines. 23:15 - What survives the paper: Why the conceptual relocation — treat the workspace as something to defend, and never let a stranger's note become your agent's rule — outlasts the provisional metrics. Recommended Reading: - Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection: The foundational treatment of indirect prompt injection — the single-shot attack this episode argues frontier models now shrug off, setting up the persistence reframe. (https://arxiv.org/abs/2302.12173) - Defeating Prompt Injections by Design (CaMeL): The data-flow defense the episode singles out as the strongest baseline, whose notion of provenance gets it 'halfway to the right idea' but stops short of persistent state. (https://arxiv.org/abs/2503.18813) - AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents: One of the two standard benchmarks the authors run to show single-shot injection now fails, motivating their multi-step ClawTrojan chains. (https://arxiv.org/abs/2406.13352) - InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents: The second benchmark used as the near-zero baseline, illustrating the gap between obvious single-context injection and the smeared-across-time attack this episode centers on. (https://arxiv.org/abs/2403.02691)