← 🖥️ ML Infrastructure & Serving
Core
GPU Memory and VRAM
VRAM is the budget that decides which models you can actually run. It is spent on three things: model weights, the KV cache, and activations. Knowing the back-of-envelope arithmetic (a 7B model at fp16 is roughly 14GB of weights) is what separates a candidate who has deployed an LLM from one who has only read about it.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
LESSONS THAT TEACH THIS
PRACTICE THIS IN REAL QUESTIONS
LLM & GenAI FundamentalsWalk me through what actually happens when an LLM generates a response.→ML Infrastructure & GPUsExplain how the CUDA execution model maps to hardware, grids, blocks, warps, SMs.→ML Infrastructure & GPUsWalk me through the GPU memory hierarchy, registers, shared memory, L2, HBM. What lives where and why?→ML Infrastructure & GPUsWhat is warp divergence and why does it hurt performance?→ML Infrastructure & GPUsExplain memory coalescing and shared-memory bank conflicts. How would you fix a kernel that has both?→ML Infrastructure & GPUsHow do you determine whether a kernel is memory-bound or compute-bound?→