GO TO A SECTION

↑ ↓ to navigate · Enter to openExplore all sections →
FDEInterviews logo
← ML Infrastructure & GPUs / 32
hardOpenAIDeepSeekTogether AI

Serve a 400B-class MoE under a 200ms p99 inter-token SLA. How do you lay it out?

A sparse model with few active params per token, a brutal latency target, and an all-to-all on the critical path. The parallelism layout that hits the SLA, why expert parallelism not tensor parallelism is the lever, and the tail the all-to-all hides.

Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.

A sparse model with few active params per token, a brutal latency target, and an all-to-all on the critical path. The parallelism layout that hits the SLA, why expert parallelism not tensor parallelism is the lever, and the tail the all-to-all hides.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 548 remaining answers · ₹2,000 / $25

Every answer, concept and course, every paper breakdown, all published video lessons, hands-on FDE Lab missions, Premium PDF guides and companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.

6 months · One payment · No auto-renewal

Study alongside free video lessons.

Keep practising

Next in this trackPlan a year of GPU fleet capacity and cost for a growing inference business. Buy, reserve, or burst?PremiumNext in this trackDesign the failure domains for a 16k-GPU training cluster. What's the blast radius of a single fault?PremiumNext in this trackLong-context traffic is OOM-ing your serving fleet at 3am. Triage the KV-cache blowup.Premium
FEDITOR'S NOTE

The screen is whether you reason from active parameters and the all-to-all, not total parameters. Candidates who size a 400B MoE like a 400B dense model are off by an order of magnitude; the strong answer uses expert parallelism to spread the experts, keeps attention TP small inside NVLink, and treats the dispatch/combine all-to-all as the latency term that actually threatens the p99.

DISCUSSION · 0

No comments yet — be the first to share your approach.