DeepSeek ML Infrastructure & GPUs interview questions

A sparse model with few active params per token, a brutal latency target, and an all-to-all on the critical path. The parallelism layout that hits the SLA, why expert parallelism not tensor parallelism is the lever, and the tail the all-to-all hides.

A very large sparse model, hundreds of experts, and users on every continent. The layout that keeps experts close to compute, routes around regional skew, and decides what 'global KV cache' can and cannot mean.

The NVIDIA warm-up that exposes whether you've written a kernel or just called .cuda(). The mapping from software hierarchy to silicon explains every GPU performance conversation that follows.

Kernel optimization is mostly the art of feeding 100+ TFLOPs from the small fast end of this hierarchy. Know the rough latencies and bandwidths, interviewers listen for the numbers.

Everyone says 'maximize occupancy.' The candidates who get hired at NVIDIA know when 25% occupancy beats 75%, and can explain the resource math that decides it.

A reported OpenAI phone-screen exercise that looks trivial and is graded on edge cases: money-safe arithmetic, billing increments, unknown SKUs, and the clarifying questions you ask before typing.

The two access-pattern killers behind most 'my kernel is 10x slower than cuBLAS' mysteries. The matrix transpose walks straight into both, which is why NVIDIA loves asking about it.

A reported DeepMind research-engineer question. The winning answer opens with a memory budget in bytes, not a list of parallelism buzzwords, here's the full arithmetic and the layout it forces.

One formula, 2 × layers × KV heads × head_dim × bytes, unlocks every LLM serving capacity question. Worked through for Llama-70B, plus the mitigation stack from GQA to paging.

The roofline model in one ratio: FLOPs per byte against the hardware's ridge point. Get the H100 arithmetic right and you can classify any kernel, including why LLM decode will never be compute-bound at batch 1.

A staple NVIDIA screen with a precise mechanical answer, and a follow-up trap about which branches actually diverge that catches most candidates.

Sounds like small talk; it's actually a depth probe with follow-ups pre-loaded. A scoping framework that works whether you've run 4 GPUs or 4,000, and the bluff that ends interviews.

The backbone question of every frontier-lab infra loop. The answer that scores is organized around what each strategy communicates and how often, not just what it splits.

The one derivation every GPU-infra loop expects on a whiteboard: scatter-reduce plus all-gather, 2(N−1)K/N per GPU, and why that's provably near-optimal. Plus the latency catch that motivates tree algorithms.

One synchronous job, 16k GPUs, and any single component can stall all of them. The blast-radius analysis interviewers want: which faults take down a rack vs the run, where the shared single points of failure hide, and how parallelism layout maps onto failure domains.

A real infra-loop question that candidates fumble by guessing. The correct map, TP on NVLink, PP and DP across the scale-out fabric, falls out of three traffic profiles, and the reasoning is the answer.

NCCL's topology tricks, GPUDirect RDMA, and the single most misleading metric in distributed training. If you've ever trusted nvidia-smi on a slow run, this question was written for you.