hardOpenAIDatabricksGoogle

The customer needs this classifier under a 150ms p99 and a $20k/month budget at 50M calls/day. LLM or not?

A budget-constrained design problem, not an ideology question. The per-call math that rules an LLM in or out, the cascade that keeps quality while cutting cost 90%, and the SLO arithmetic that decides where the LLM is even allowed to sit.

Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 548 remaining answers · ₹2,000 / $25

Every answer, concept and course, every paper breakdown, all published video lessons, hands-on FDE Lab missions, Premium PDF guides and companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.

6 months · One payment · No auto-renewal

Study alongside free video lessons.

Keep practising

Next in this trackThe customer's marketplace has network effects, so a user-level A/B test is biased. How do you measure the model's impact?Premium Next in this trackThe model aced every offline eval and the customer says it's useless. The labels look fine. What now?Premium Next in this trackThe customer wants to test five model variants without losing money on the bad ones. A/B test or a bandit?Premium

FEDITOR'S NOTE

The screen is whether you do the arithmetic before picking a side: convert the monthly budget and daily volume into a per-call ceiling, compare it to realistic LLM and classical costs and latencies, and only then design. The reserved follow-up is the cascade, a cheap high-coverage model that handles the easy 90% and escalates the uncertain tail to the LLM, which is how you respect both constraints at once; candidates who answer 'LLM' or 'classical' without the budget math fail the question on contact.

DISCUSSION · 0

No comments yet — be the first to share your approach.