Why Do Multi-Agent LLM Systems Fail?

Multi-agent systems split a job across several LLM agents with roles: a planner and a coder, a CEO and a reviewer, a supervisor and a set of tool agents. They are popular, and they often disappoint. The authors measured failure rates between 41% and 86.7% across six open-source systems, each on its own benchmark.

The question the paper asks is why. The team, a Berkeley group including Matei Zaharia, Joseph Gonzalez and Ion Stoica, took over 150 execution traces from five frameworks (traces average over 15,000 lines each) and had six experts read them with Grounded Theory, a qualitative method in which categories come out of the data rather than going in as hypotheses. They kept reading until new traces stopped producing new failure types. Three annotators then labeled traces independently and argued out their disagreements over three rounds until agreement reached Cohen's κ = 0.88.

The result is MAST, the Multi-Agent System Failure Taxonomy: 14 failure modes in three groups. System design issues include ignoring the task's constraints, repeating steps and not knowing when to stop. Inter-agent misalignment covers agents that go off task, withhold what another agent needs, or say one thing and do another. Task verification failures are stopping early, or checking badly or not at all.

To scale up, they calibrated an LLM judge (OpenAI's o1 with few-shot examples) against the human labels, reaching 94% accuracy and κ = 0.77, and used it to label 1,642 traces from seven frameworks. Their central claim is that these failures mostly come from organizational design rather than model weakness, so better models alone will not fix them.

When a customer's multi-agent pilot stalls, the conversation usually turns to swapping the model. This paper gives you a better first move: read the traces and name the failure.

A shared vocabulary speeds up debugging. "The agents get confused" is not actionable. "The reviewer passes code that compiles but never runs it" (no or incomplete verification) or "the tool agent knows the username must be a phone number and never says so" (information withholding) points straight at the fix.

Most of the fixes are design work, which is FDE work. Clear role definitions, explicit stop conditions, who is allowed to end the conversation, and a verifier that checks the actual goal rather than whether the code compiles are all choices you control without retraining anything.

It is also an argument for fewer agents. If a pipeline of fixed steps or a single agent with tools would do the job, the 14 failure modes are 14 reasons not to add coordination you do not need. Several of them, such as conversation reset, information withholding and ignoring another agent's input, cannot happen to one agent.

Use the error bars. The interventions in the paper improved one system by five tasks out of 32. Before you tell a customer a change fixed their system, make sure your own evaluation is big enough to show it.