attention

Modern LLMs share one architectural skeleton. Learn the five components and you can read almost any model card cold, plus the single distinction that proves to an interviewer you actually understand it rather than the buzzwords.

The same operation scores attention and ranks your RAG results. Knowing exactly what the dot product measures, and when its magnitude term quietly breaks your retrieval, is what separates a working pipeline from a mysteriously bad one.

Rotary embeddings encode position as a rotation of the query and key vectors, so attention scores depend only on the gap between two tokens. That relative property is why RoPE generalizes past its training length where learned absolute tables fall off a cliff.

Attention is bottlenecked by moving the N x N score matrix to and from HBM, not by FLOPs. FlashAttention tiles the computation in fast on-chip SRAM and fuses softmax with an online running statistic, so it computes the exact same output while never materializing the full matrix.

softmax(QKᵀ/√d_k)·V in four lines, but the grade is the three details people drop: the √d_k scale, subtracting the row max for numerical stability, and getting the causal mask to add -inf before the softmax, not after.

The answer is not just 'transformers are better.' It is two specific wins: every position can attend to every other in one step (long-range dependencies) and the whole sequence trains in parallel instead of one token at a time. Plus the cost that buys, and where RNNs still make sense.