IN THIS BREAKDOWN · 10 SECTIONS
You can read 17% of this breakdown.
The paper in one minute
Suppose two language models make different mistakes. Looking at their completed answers, you can choose the correct response more often than either model succeeds alone. That does not mean a router can make the same choice before either model runs.
The Routing Plateau studies this gap. Across 21 routing methods and five benchmarks, the stronger methods achieve similar scores despite their different designs. A nearest-neighbor baseline is competitive with learned routers. The authors argue that predicting which particular model will succeed from the query alone is a shared bottleneck.
More training data, a larger encoder and end-to-end fine-tuning improve the evaluated routers, but leave much of the gap to a hindsight oracle intact. The result is an empirical finding about these model pools and experiments, not a theorem that better routing is impossible.
For an FDE, the useful question changes from “Which router is newest?” to “What evidence could let this router make a better decision on our requests?” The answer may involve a first attempt, a validator or a task-specific feature, each with its own cost.
Research notes: the problem, approach and results
routing methods in the comparison
benchmark datasets
mean improvement across eight scaled and fine-tuned routers
Different router designs reach similar quality despite unused potential in the model pool.
Compare 21 routers under common data, encoders and evaluation, then test scaling interventions.
Top routers cluster closely; stronger training helps modestly without closing the oracle gap.
Why it matters to an FDE
A routing project often starts with a promising cost-quality spreadsheet. Several inexpensive models work well on some tasks, while a larger model handles others. The spreadsheet describes available outcomes. Production routing must select a model using information available before the call, under latency, privacy and budget constraints.
If the observable request features do not predict those differences reliably, a complex router can reproduce a simple global ranking at greater operational cost. Conversely, a narrow customer workflow may expose strong signals that broad benchmark embeddings miss.
Use model routing and enterprise model selection to frame a local evaluation. Compare the best fixed model, a cheap fixed model, a simple router and any proposed learned system. Keep an oracle as a diagnostic bound, with a clear warning that it uses answers unavailable at decision time.
Understand the mechanism. Try it on a real problem.
Unlock this breakdown and 199 more papers.
Premium opens every paper in full, plus every answer, concept, course and guide on the site. One payment for six months of access.
6 months · One payment · No auto-renewal
- ◇1 architecture diagram
- </>2 runnable code blocks
- ▦3 results tables
- 03Key takeaways
Six checks for interpreting router comparisons and deciding what information to collect next
Free account - 04How it works
Follow the shared outcome matrix, controlled router comparisons and scaling interventions
Premium - 05The numbers that matter
Read the close router scores and scaling gains without confusing an oracle with a service
Premium - 06Try it yourself
Enumerate a tiny routing problem and see why an oracle's accuracy is not a trained policy
Premium - 07Where it breaks
Identify the limits of fixed model pools, recorded outcomes and benchmark-level averages
Premium - 08Using it on a deployment
Build a routing evaluation that separates feasible candidates, predictive signal and operating cost
Premium - 09Interview angles
Explain why model diversity does not automatically produce a useful router
Premium - 10Read next
Connect the plateau finding to practical routing methods and budget-aware inference
Premium