Core
Knowledge Distillation
Distillation trains a small student model to mimic a large teacher, learning from the teacher's full output distribution rather than just hard labels. The soft targets carry extra signal about how the teacher 'thinks', so the student keeps much of the quality at a fraction of the size and latency. Knowing when distillation beats quantization or pruning is standard FDE ground when you have a latency or cost budget to hit.
a free account unlocks the core curriculum tier · no card
Related concepts
Practise this in questions
LLM & GenAI FundamentalsYou have a fixed latency and cost budget at scale. Walk me through quantization vs distillation to hit it. →hard · PremiumLLM & GenAI FundamentalsWalk me through what actually happens when an LLM generates a response. →easy · FreeML Infrastructure & GPUsYour training run compiles and runs, but the loss is flat. Walk me through your debugging process. →hard · Free with sign-inLLM & GenAI FundamentalsA customer says your LLM app is too slow. Give me five levers to reduce latency, and their tradeoffs. →medium · Free with sign-inML Infrastructure & GPUsWhat is the KV cache, why does it dominate serving memory, and how do you size it? Do the math for a 70B model. →medium · Free with sign-inML Infrastructure & GPUsHow does vLLM's PagedAttention work, and what problem does it actually solve? →medium · Free with sign-in