long context

Everyone can define it; few candidates know why 'advertised context' and 'usable context' aren't the same thing. The cost, latency, and attention caveats that make this answer stand out.

Million-token windows didn't kill RAG; they changed when you reach for it. The decision rule that holds up in production, and the silent failure each option hides behind a confident answer.

Rotary embeddings encode position as a rotation of the query and key vectors, so attention scores depend only on the gap between two tokens. That relative property is why RoPE generalizes past its training length where learned absolute tables fall off a cliff.

The fleet was fine on 8k contexts and falls over when a few 200k-token requests arrive. The math that explains why, the admission and pooling fixes that contain it, and why throwing GPUs at it is the wrong first move.

Going 4K to 1M is not one trick. It is RoPE rescaling chosen by frequency band, a long-doc continued-pretrain mix, needle evals that actually catch failures, and KV math that forces ring attention.