Why Multi-Agent LLM Systems Fail: A Comprehensive Study

This paper, "Why Do Multi-Agent LLM Systems Fail?", presents a comprehensive study into the shortcomings of systems where multiple large language model agents collaborate. Through extensive analysis of several popular multi-agent frameworks across numerous tasks, the authors identify and categorize 14 distinct failure modes into three main areas: specification/design flaws, inter-agent misalignment, and issues with task verification/termination. To facilitate further research, they introduce MASFT, the first structured failure taxonomy for these systems, along with a scalable LLM-based evaluation pipeline and an open-sourced dataset of annotated failure traces. The study also explores potential interventions, revealing that simple fixes are insufficient, highlighting the need for fundamental redesigns inspired by high-reliability organizations to build more robust multi-agent LLM systems.