The fleet was fine on 8k contexts and falls over when a few 200k-token requests arrive. The math that explains why, the admission and pooling fixes that contain it, and why throwing GPUs at it is the wrong first move.
Updated Sep 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.
The fleet was fine on 8k contexts and falls over when a few 200k-token requests arrive. The math that explains why, the admission and pooling fixes that contain it, and why throwing GPUs at it is the wrong first move.
The screen is whether you connect KV-cache math to a production incident and reach for admission control and pooling before capacity. Strong candidates compute that a single 200k-context request can dwarf dozens of chat requests in KV, explain that one giant sequence starves the shared block pool, and propose token-shape-aware admission plus a separate long-context tier rather than just adding GPUs.
No comments yet — be the first to share your approach.
