FDEInterviews logo
System Design & Production Engineering / 38
hardOpenAIAnthropicDatabricks

Hit a hard p99 SLA on an LLM product without blowing a fixed monthly spend ceiling. Model it.

Latency, cost, and quality are one budget with three claims on it. The strong answer treats the p99 SLA and the spend ceiling as a joint constraint, finds where they fight (batching), and names the lever it pulls when traffic exceeds what the ceiling can buy at SLA.

Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.

Latency, cost, and quality are one budget with three claims on it. The strong answer treats the p99 SLA and the spend ceiling as a joint constraint, finds where they fight (batching), and names the lever it pulls when traffic exceeds what the ceiling can buy at SLA.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 523 remaining answers · ₹2,000 / $25
UP NEXT ON YOUR JOURNEY
FEDITOR'S NOTE

This screens for whether someone can hold two hard constraints at once instead of optimizing one and apologizing for the other. The trap is batching: it is the biggest cost lever and the biggest p99 risk, and a candidate who reaches for aggressive continuous batching to cut cost without naming the tail-latency hit has not run a serving fleet. The discriminating follow-up is 'traffic doubles mid-month and you are on track to blow the ceiling, what gives?' The answer that scores names the explicit degradation policy decided in advance (route more traffic to the cheap model, shrink max output tokens, shed or queue low-priority work) rather than silently breaching the SLA or the budget. Anyone who says 'autoscale' without a unit-cost number is guessing.

DISCUSSION · 0

No comments yet — be the first to share your approach.