Core

Quantization

Quantization stores model weights (and sometimes activations) in fewer bits, fp16 down to int8 or 4-bit, which cuts memory and speeds inference. The quality hit is usually small at int8 and larger at 4-bit. Knowing post-training quantization versus quantization-aware training, and when each is acceptable, is standard FDE interview ground.

a free account unlocks the core curriculum tier · no card

RELATED CONCEPTS

GPU Memory and VRAM Inference Serving (vLLM, TGI)LoRA and Parameter-Efficient Fine-tuning

LESSONS THAT TEACH THIS

Hard DeploymentsBudget weights, resident cache and runtime memory in consistent units30 min→Hard DeploymentsMeasure throughput without losing the requests that failed180 min→Hard DeploymentsTest the model server's cache, adapter and shared-failure boundaries30 min→

PRACTICE THIS IN REAL QUESTIONS

LLM & GenAI FundamentalsWalk me through what actually happens when an LLM generates a response.→ML Infrastructure & GPUsWalk me through the GPU memory hierarchy, registers, shared memory, L2, HBM. What lives where and why?→ML Infrastructure & GPUsExplain memory coalescing and shared-memory bank conflicts. How would you fix a kernel that has both?→ML Infrastructure & GPUsWhat do ZeRO and FSDP actually shard, and how much memory does each stage save? Where does gradient checkpointing fit?→ML Infrastructure & GPUsWe need to train a 100B-parameter model that won't fit in memory. Design the data and model parallelism.→LLM & GenAI FundamentalsA customer says your LLM app is too slow. Give me five levers to reduce latency, and their tradeoffs.→