KV-caching in autoregressive decoding: what does it store, why does it cut latency, and how does its memory cost scale?

The screen is whether the candidate knows the cache stores per-layer attention keys and values for all prior tokens (not the activations, not the logits), why that turns per-token cost from re-processing the whole sequence into a single new-token step, and the scaling law: memory ~ 2 * batch * seq_len * layers * kv_heads * head_dim * bytes. The reserved follow-up: 'so how do you shrink it' (GQA/MQA, quantizing the cache, paged attention). Saying it 'caches the output' or forgetting the linear growth with sequence and batch is the tell.