Pinpointing the Culprit: How Researchers Are Automating Failure Attribution in Multi-Agent LLM Systems
Introduction
Large Language Model (LLM) multi-agent systems have gained significant traction for their ability to jointly tackle complex problems. Yet, despite the flurry of activity between agents, these systems frequently fail. Developers then face a critical challenge: identifying which agent caused the failure and at what stage. Sifting through massive interaction logs is akin to finding a needle in a haystack—a slow, labor-intensive process that hinders development and optimization.
To address this, researchers from Penn State University and Duke University, in collaboration with Google DeepMind, University of Washington, Meta, Nanyang Technological University, and Oregon State University, have introduced a new research problem: "Automated Failure Attribution." They built the first benchmark dataset, Who&When, and developed multiple automated attribution methods. Their work not only highlights the complexity of the task but also paves the way toward more reliable LLM multi-agent systems. The paper was accepted as a Spotlight presentation at ICML 2025, and the code and dataset are now fully open-source.
The Growing Complexity of Multi-Agent Systems
Why Failures Happen
LLM-driven multi-agent systems are powerful yet fragile. A single agent’s mistake, a misunderstanding between agents, or an error in information transmission can cause the entire system to fail. As these systems become more autonomous and involve longer chains of reasoning, diagnosing failures becomes exponentially harder.
Currently, developers rely on two inefficient methods:
- Manual Log Archaeology: Developers must painstakingly review lengthy interaction logs to locate the failure source.
- Heavy Expertise Dependence: Successful debugging requires deep understanding of both the system architecture and the specific agents involved.
These approaches are time-consuming and often impractical for complex systems, creating a pressing need for automated solutions.
The Who&When Benchmark
Dataset Construction
To enable automated failure attribution, the team created Who&When, the first benchmark dataset specifically for this task. It includes diverse failure scenarios from multi-agent systems across different domains. Each instance records the interaction logs, the failure outcome, and ground truth labels indicating which agent failed and when.
Key Features
The dataset covers multiple types of failures, such as incorrect reasoning, miscommunication, and incomplete task execution. It also varies the number of agents and the length of interactions, allowing researchers to test attribution methods under realistic conditions.
Automated Failure Attribution Methods
Evaluation and Results
The researchers developed and evaluated several automated attribution methods. These include statistical approaches that analyze agent contributions, as well as learning-based models that use the interaction logs to predict failure sources. Initial results show that automated methods can significantly reduce debugging time while maintaining high accuracy compared to manual analysis. However, the task remains challenging, especially for subtle failures where multiple agents are involved.
Implications and Future Work
Impact on Reliability
By automating failure attribution, developers can quickly iterate on system designs, fix problematic agents, and improve overall reliability. This work opens the door to more robust multi-agent systems that can self-diagnose and recover from errors.
Future research may extend the benchmark to include dynamic environments and explore integration with real-time monitoring tools. The open-source release of Who&When and the associated code enables the broader community to build upon these foundations.
Available Resources
The paper, code, and dataset are publicly available:
- Paper: arXiv
- Code: GitHub
- Dataset: Hugging Face