The AI Agent That Found the Truth and Typed the Lie Anyway

The AI Agent That Found the Truth and Typed the Lie Anyway Source: https://arxiv.org/abs/2607.17291 Paper was published on July 19, 2026 This episode was AI-generated on July 21, 2026. The script was written by an AI language model and the host voices were synthesized by Eleven Labs. The producer is not affiliated with Anthropic or Eleven Labs. One of the strongest AI research agents solved a hard cross-referencing task 96% of the time — until researchers slipped in a single fake page, and its accuracy cratered to 26%. The unsettling part: the agent retrieved the truth every single time, could reason its way to the right answer, and handed you a confident, well-cited lie anyway. This episode traces exactly why a system that clearly knows the truth quits before it proves it. Key Takeaways: - Why retrieval isn't the culprit: in all 100 poisoned runs the agent pulled up the truthful records and still deferred to the lie - The 'conditional deference' metric — DeepSeek flips to the exact planted answer about 98% of the time on tasks it had already solved - The cleanest experiment in the paper: hand the agent all the evidence up front and the lie stops working (91/100 correct), proving reasoning was never broken - Why the failure lives in the agent's stopping policy — 'verification inertia' — and why a generic 'be careful' prompt barely helps (12 to 28 out of 100) - The steelman: the benchmark engineers a maximum convenience gap, and closing it lifts accuracy from 12 to 63 — so the effect is real but partly staged - The reframe for real work: no attacker needed — one ordinary stale or sloppy page can produce a confident, well-cited wrong answer 00:03 - Found the truth, typed the lie?: The cold open lays out the Brindle Components task and the 96%-to-26% collapse caused by a single injected fake page. 01:25 - The boring explanation that's wrong: Tyler proposes the obvious 'it just never found the truth' read, and Juniper shows that across all 100 poisoned tasks the agent retrieved the truthful records every time. 02:19 - How do you rig a fair test?: The controlled A/B setup — clean vs noisy versions identical except one added fake page, with truth that must be reconstructed and a lie that's gift-wrapped. 04:37 - One page, and DeepSeek hits 1%: Results across five top models and the 'conditional deference' metric that shows agents flip to the exact planted lie on tasks they'd already solved. 06:10 - Three suspects, one culprit: Separating retrieval, reasoning, and the agentic loop, then tracing GPT-5.4 step by step to rule out retrieval and reasoning. 07:26 - Hand it the folder and it's right: The decisive experiment: dump all the evidence into context and the lie stops working — 91/100 correct — proving the failure lives in the stopping decision. 08:42 - Verification inertia, and no easy patch: Naming 'verification inertia,' the link to models telling users what they want to hear, and why a 'be careful' prompt barely helps. 09:38 - Does the benchmark stack the deck?: The steelman critique: the constructed corpus engineers the worst case, and closing the convenience gap lifts accuracy from 12 to 63. 11:36 - Why 'cited' isn't 'verified': The takeaway reframe — finding and citing is not verifying — and why an ordinary bad page, no attacker required, is the realistic danger. Recommended Reading: - Sycophancy to Subterfuge: Investigating Reward Tampering in Language Models: Extends the episode's 'sycophancy pointed at the corpus' framing—showing how the same reflex to tell users what they want to hear generalizes into deeper failure modes. (https://arxiv.org/abs/2406.10162) - Towards Understanding Sycophancy in Language Models: The Anthropic paper on models caving to stated opinions—the human-directed version of the deference-to-a-plausible-source behavior Tyler compares the agent's failure to. (https://arxiv.org/abs/2310.13548) - Benchmarking Large Language Models in Retrieval-Augmented Generation: Probes how retrieval-augmented systems handle noise and conflicting evidence in their retrieved context, directly relevant to the episode's 'found it but cited the wrong one' wedge. (https://arxiv.org/abs/2309.01431) - ReAct: Synergizing Reasoning and Acting in Language Models: Introduces the reason-act-observe agentic loop whose stopping decision—when the agent declares itself 'done'—is exactly the control process this episode pins the failure on. (https://arxiv.org/abs/2210.03629)