IN THIS BREAKDOWN · 10 SECTIONS
You can read 19% of this breakdown.
The paper in one minute
Multi-agent systems are popular, and so are papers saying they do not help. This one tries to settle when they do, by changing only the coordination structure.
The authors compare five architectures. A single agent runs one loop. Independent agents work in parallel and their outputs are concatenated, with no checking. Decentralized agents debate peer to peer. Centralized sub-agents report to an orchestrator that decomposes the task and checks results. Hybrid adds peer messages to an orchestrator. Every configuration gets the same prompts, the same tools and a matched total token budget (about 4,800 tokens per trial on average), so a team of agents splits the budget a single agent spends alone.
They run 260 configurations: nine models from OpenAI, Google and Anthropic (Claude Sonnet 3.7 dropped out of two benchmarks after deprecation), on six benchmarks chosen because they need real interaction: BrowseComp-Plus (web research), Finance-Agent, PlanCraft (Minecraft crafting plans), Workbench (business tools), SWE-bench Verified and Terminal-Bench.
Results depend heavily on the task. On Finance-Agent, where analysis splits naturally into news, filings and operations, centralized coordination improved results by 80.8%. On PlanCraft, where each step depends on the last, every multi-agent architecture lost, by 39% to 70%. Elsewhere, effects were small. Across everything, the average change was -0.3%.
They then fit a regression to explain these effects and derive rules: stay single-agent when one agent already scores above about 45%; expect tool-heavy tasks to suffer from coordination overhead; prefer architectures with a central checker to limit error propagation.
Research notes: the problem, approach and results
best and worst multi-agent results against one agent: centralized on Finance-Agent, independent on PlanCraft
mean relative change from adding agents across all six benchmarks and four architectures
average turns per task for the hybrid architecture against a single agent, under a matched token budget
Teams add agents (orchestrators, debaters, parallel workers) because it seems to help, but published comparisons change prompts, tools and compute at the same time as the architecture, and mostly test on static benchmarks where ensembling is easy. Nobody could say when coordination pays on real agentic tasks.
Hold everything fixed except coordination. Run a single agent and four multi-agent topologies (independent, decentralized, centralized, hybrid) with identical prompts and tools and matched token budgets, across nine models from three families and six agentic benchmarks, then fit a regression on model capability, task properties and coordination metrics.
Effects ranged from +80.8% (centralized, Finance-Agent) to -70.0% (independent, PlanCraft); the overall mean was -0.3%. Multi-agent setups tended to lose when the single agent already scored above about 45%. The regression picked the best architecture for 87% of held-out configurations, but its R² was 0.37 to 0.41, and the paper's own robustness checks leave capability saturation as the one firm result.
Why it matters to an FDE
Customers often ask for "a team of agents" because it sounds capable. This paper gives you evidence to push back, and to say yes when it fits.
Start with one agent and measure it. If a single agent with good tools already handles most cases, the paper found adding agents tends to hurt. The capability-saturation effect is the one finding that survived every robustness check the authors ran.
Add agents when the task splits into independent streams. The large gains came where subtasks could run separately and be merged: financial analysis across news, filings and operations. Sequential tasks, where each step changes the state the next depends on, got worse with every multi-agent design.
Budget for coordination. Under a fixed token budget, the hybrid architecture took 44.3 turns on average against 7.2 for one agent, and successes per thousand tokens fell from 67.7 to 13.6. If budget is not fixed, coordination costs money instead of accuracy.
Put a checker in the middle. Independent agents whose outputs are concatenated without review did worst overall. If you parallelize, have something verify and merge.
Read the paper's precise-looking numbers with care. The thresholds come from a regression that explains about 40% of variance, several of its headline effects are not significant under its own checks, and two benchmarks used only 20 tasks per configuration.
Understand the mechanism. Try it on a real problem.
Unlock this breakdown and 149 more papers.
Premium opens every paper in full, plus every answer, concept, course and guide on the site. One payment for six months of access.
6 months · One payment · No auto-renewal
- ◇1 architecture diagram
- </>1 runnable code experiment
- ▦2 results tables
- ?5 interview questions with answers
- 03Key takeaways
Results by benchmark, the patterns that hold, the ones that do not, and three misreadings
Free account - 04How it works
The five topologies, the controls, the coordination metrics and the regression
Premium - 05The numbers that matter
Per-benchmark means, the architecture table, the regression and its robustness checks
Premium - 06Try it yourself
Recompute the headlines, test what the metrics can carry, and size the noise of a 20-task cell
Premium - 07Where it breaks
Architecture-level metrics, weak robustness, small samples, fixed prompts and internal slips
Premium - 08Using it on a deployment
A decision procedure for single versus multi-agent, and how to run the comparison yourself
Premium - 09Interview angles
Five questions this paper prepares you for, with answer hints
Premium - 10Read next
Where to go from here on this site
Premium