Challenge · intermediate · 25 min · Premium
Does it fit
Northwind wants the claims model on their own two 80 GB cards; the sheet says 70 GB of weights. Work out whether it fits with a KV cache at their concurrency, in the units the card uses, and set the admission limit that keeps it fitting.
Brief
- Customer
- Northwind Insurance (synthetic customer; the model and hardware figures on this box are illustrative and made up for the exercise)
- Situation
- Northwind's platform team has two 80 GB accelerators and a vendor sheet for a 35-billion-parameter model at 16-bit weights. They want 32 concurrent claims conversations at an 8k context. Their question is one sentence: does it fit, and if not, what do we cut.
- Your role
- You are the FDE sizing the deployment. You have the vendor sheet, the sizing tool on this box, and the serving config (max concurrency) that the tool checks against.
- Goal
- Convert the weights to the unit the card reports, compute the KV cache at the requested concurrency, decide whether it fits, and set an admission limit that does.
Objectives
- 1.State the weights in GiB (you commit to an answer)
- 2.State the KV cache at the requested concurrency (you commit to an answer)
- 3.Say whether the requested deployment fits (you commit to an answer)
- 4.Set an admission limit that fits with headroom
Mission workspace
This mission is part of premium, with every challenge and incident in the lab. The brief above is the whole problem; the box, the hints and the debrief are what you unlock.
Get full Premium access · ₹2,000 / $25
Every answer, concept and course, plus Premium PDF guides, companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.
6 months · One payment · No auto-renewal
Study alongside free video lessons.