vllm
FDE interview questions tagged vllm, across every topic.
2 questions · 0 unlocked for you
Concepts behind "vllm"
The curriculum that explains the ideas these questions test.
Core
Inference Serving (vLLM, TGI)Serving LLMs at high throughput under a latency SLO is its own engineering problem. Continuous batching keeps the GPU busy across requests of different lengths, and PagedAttention stops the KV cache from wasting memory. Naive one-request-at-a-time serving leaves most of an expensive GPU idle, which is why purpose-built runtimes like vLLM and TGI exist.🖥️ ML Infrastructure & ServingSign in
Advanced
PagedAttentionAllocating each sequence's KV cache as one contiguous block forces you to reserve space for the maximum possible length, which fragments GPU memory and wastes most of it. PagedAttention borrows OS-style paging: it stores the KV cache in fixed-size non-contiguous blocks tracked by a per-sequence block table, so memory is packed, grown on demand, and shareable across sequences. It is the core memory trick that lets vLLM pack far more concurrent sequences into the same VRAM.🖥️ ML Infrastructure & Serving🔒 Premium