The attacker moves second: stronger adaptive attacks bypass defenses against LLM jail- Breaks and prompt injections

The academic paper discusses the critical flaws in current methods used to evaluate the robustness of large language model (LLM) defenses against jailbreaks and prompt injections. The authors argue that testing defenses with static or computationally weak attacks yields a false sense of security, as demonstrated by the fact that they successfully bypassed twelve different recent defenses with an attack success rate exceeding 90% in most cases. Instead, they propose that robustness must be measured against adaptive attackers who systematically tune and scale optimization techniques, including gradient descent, reinforcement learning, search-based methods, and human red-teaming. The paper emphasizes that human creativity remains the most effective adversarial strategy, and future defense work must adopt stronger, adaptive evaluation protocols to make reliable claims of security.