FDEInterviews logoFDE/Interviews

DeepSeek ML Infrastructure & GPUs interview questions

ML Infrastructure & GPUs is a core part of the DeepSeek AI & ML Engineer loop. GPU/TPU workloads, distributed training and parallelism, inference serving (vLLM, batching, KV cache), cluster scheduling and scaling API gateways: the infra depth NVIDIA, Google and the AI labs probe. Below are the ml infrastructure & gpus questions to prepare, the ones tagged to DeepSeek first, then the highest-signal questions from our ML Infrastructure & GPUs track, each with an answer written to a senior-engineer bar.

ML Infrastructure & GPUs questions tagged to DeepSeek

2 questions · 0 unlocked for you
Open full answer →
Open full answer →

More ML Infrastructure & GPUs questions for DeepSeek's loop

The highest-signal ml infrastructure & gpus questions candidates rate most useful, modeled on what DeepSeek's AI & ML Engineer loop tests.

15 questions · 10 unlocked for you
Open full answer →
Open full answer →
Open full answer →
Open full answer →
Open full answer →
Open full answer →
Open full answer →
Open full answer →
Open full answer →
Open full answer →
Open full answer →
Open full answer →
Open full answer →
Open full answer →

Concepts behind DeepSeek's ML Infrastructure & GPUs round

The vocabulary and mental models these questions assume. Start with the foundations free; the deeper, interview-defining ideas are part of premium.

Core
Sign in
GPU Memory and VRAMVRAM is the budget that decides which models you can actually run. It is spent on three things: model weights, the KV cache, and activations. Knowing the back-of-envelope arithmetic (a 7B model at fp16 is roughly 14GB of weights) is what separates a candidate who has deployed an LLM from one who has only read about it.
Core
Sign in
QuantizationQuantization stores model weights (and sometimes activations) in fewer bits, fp16 down to int8 or 4-bit, which cuts memory and speeds inference. The quality hit is usually small at int8 and larger at 4-bit. Knowing post-training quantization versus quantization-aware training, and when each is acceptable, is standard FDE interview ground.
Core
Sign in
Knowledge DistillationDistillation trains a small student model to mimic a large teacher, learning from the teacher's full output distribution rather than just hard labels. The soft targets carry extra signal about how the teacher 'thinks', so the student keeps much of the quality at a fraction of the size and latency. Knowing when distillation beats quantization or pruning is standard FDE ground when you have a latency or cost budget to hit.
Core
Sign in
Inference Serving (vLLM, TGI)Serving LLMs at high throughput under a latency SLO is its own engineering problem. Continuous batching keeps the GPU busy across requests of different lengths, and PagedAttention stops the KV cache from wasting memory. Naive one-request-at-a-time serving leaves most of an expensive GPU idle, which is why purpose-built runtimes like vLLM and TGI exist.
Advanced
🔒 Premium
Continuous BatchingStatic batching runs a fixed group of requests to completion together, so a batch of one short reply and one long reply makes the GPU idle while it waits on the longest. Continuous batching adds and evicts sequences from the running batch every decode step, keeping the GPU saturated and multiplying throughput. It is the scheduling trick at the heart of vLLM and every modern LLM serving stack.
Advanced
🔒 Premium
PagedAttentionAllocating each sequence's KV cache as one contiguous block forces you to reserve space for the maximum possible length, which fragments GPU memory and wastes most of it. PagedAttention borrows OS-style paging: it stores the KV cache in fixed-size non-contiguous blocks tracked by a per-sequence block table, so memory is packed, grown on demand, and shareable across sequences. It is the core memory trick that lets vLLM pack far more concurrent sequences into the same VRAM.
Advanced
🔒 Premium
Speculative DecodingDecoding is slow because each token needs a full forward pass through a huge memory-bound model. Speculative decoding has a small fast draft model propose several tokens at once, then the large target model verifies them all in a single forward pass and keeps the longest correct prefix. A careful accept rule makes the output provably identical to sampling from the target model, so you get lower latency for free, not an approximation.
Advanced
🔒 Premium
Distributed Training (FSDP, Parallelism)When a model or its training state will not fit on one GPU, you split the work across many. Data parallelism replicates the model and splits the batch; tensor parallelism splits a single layer's math across GPUs; pipeline parallelism splits the layer stack into stages; and FSDP/ZeRO shard the parameters, gradients, and optimizer states themselves. Each buys memory by spending network bandwidth, so the real skill is composing them to fit the model while keeping the GPUs busy.
Explore the full curriculum →
DEEPSEEK ML INFRASTRUCTURE & GPUS FAQ
What ML Infrastructure & GPUs questions does DeepSeek ask in interviews?▲

DeepSeek's AI & ML Engineer loop draws ml infrastructure & gpus questions such as "Serve a 400B-class MoE under a 200ms p99 inter-token SLA. How do you lay it out?", "Design GPT-scale MoE inference as a global service across regions. How do you lay it out?", "Explain how the CUDA execution model maps to hardware, grids, blocks, warps, SMs.". GPU/TPU workloads, distributed training and parallelism, inference serving (vLLM, batching, KV cache), cluster scheduling and scaling API gateways: the infra depth NVIDIA, Google and the AI labs probe. The full set, ordered easy to hard with expert answers, is below.

How should I prepare for the DeepSeek ML Infrastructure & GPUs round?▼
Does DeepSeek hire Forward Deployed Engineers?▼
What does the DeepSeek interview test?▼

Prep the whole DeepSeek AI & ML Engineer loop

ML Infrastructure & GPUs is one round. Unlock every answer across DeepSeek's full loop, plus the concept curriculum, for 6 months. One payment, no auto-renewal. Free questions in every track to start.

Unlock all answers · ₹2,000 / $25Full DeepSeek interview guide

Independent and not affiliated with DeepSeek. All trademarks belong to their owners.