Core

KV Cache

During autoregressive decoding a model would recompute attention over every prior token at each step; the KV cache stores each token's key and value vectors so each new token only attends, never recomputes, making per-token generation far cheaper. The cache grows with sequence length times batch size and becomes the memory bottleneck in serving, which is what motivates PagedAttention. FDE loops probe it because it explains why long contexts are costly to serve and why throughput, not the model, is often the constraint.

a free account unlocks the core curriculum tier · no card

RELATED CONCEPTS

Attention and Self-Attention Inference Serving (vLLM, TGI)PagedAttention The Context Window

LESSONS THAT TEACH THIS

Hard DeploymentsBudget weights, resident cache and runtime memory in consistent units30 min→Hard DeploymentsMeasure throughput without losing the requests that failed180 min→Hard DeploymentsTest the model server's cache, adapter and shared-failure boundaries30 min→

PRACTICE THIS IN REAL QUESTIONS

LLM & GenAI FundamentalsWalk me through what actually happens when an LLM generates a response.→AI Security, Privacy & GovernanceExplain direct vs indirect prompt injection, and how each one actually breaks an LLM application.→System Design & Production EngineeringWhat do p50, p95, and p99 latency actually mean, and why do production engineers obsess over the tail?→AI Security, Privacy & GovernanceWhy do system prompts leak, and what should never be in one?→AI Security, Privacy & GovernanceWalk me through the OWASP LLM Top 10, which two risks would you prioritize for an enterprise agent deployment?→RAG & Agent System DesignHow would you evaluate a RAG system before letting a customer launch it?→