The Transformer, Intuitively
The transformer is the architecture behind every modern large language model, built on self-attention that lets each token look at every other token in parallel. FDE loops do not want the math; they want you to explain why attention beat RNNs, what decoder-only means, and why context length is expensive, in plain language an exec or a teammate can follow.
TL;DR: A transformer turns tokens into vectors, then uses self-attention so every token can pull in information from every other token, stacking that operation dozens of times to build meaning. It beat RNNs because attention processes a whole sequence in parallel instead of one step at a time, and the same all-pairs attention is why cost grows with the square of the sequence length.
The core move: attention
Before transformers, sequence models like RNNs read text one token at a time, carrying a running summary forward. That is inherently sequential, so it is slow to train and tends to forget what it saw long ago.
The transformer's idea is to drop the step-by-step recurrence and let every token look directly at every other token at once. For each token, attention asks "which other tokens should I pay attention to, and how much?" and blends their information accordingly. When the model processes "it" in "the trophy did not fit in the suitcase because it was too big," attention is what lets "it" pull meaning from "trophy" rather than "suitcase." Every token does this for every other token, which is where the all-pairs cost comes from.
How a block is built
A transformer is a stack of identical blocks. One block does roughly this:
- Token + position embeddings. Each token becomes a vector, plus a signal encoding its position (attention itself has no inherent sense of order, so position must be added in).
- Self-attention. Each token gathers a weighted mix of information from the others.
- Feed-forward network. A small per-token transformation that lets the model reshape what attention gathered.
- Residual connections and layer normalization. The block adds its output back to its input (the residual) and normalizes, which keeps gradients stable so you can stack many layers without the signal degrading.
Stack that block tens of times and you get a model that builds up meaning layer by layer, from surface patterns in the early layers to abstract relationships in the later ones.
Three shapes: decoder-only, encoder-only, encoder-decoder
The shapes differ in what each token is allowed to see:
- Encoder-only (the BERT family) lets every token see the entire sequence, past and future. That is great for understanding tasks like classification and producing embeddings, but it does not generate text left to right.
- Decoder-only (the GPT family, and most chat models) masks the future: each token can only attend to tokens before it. That constraint is exactly what makes the model good at generating the next token, one at a time. Most LLMs you interact with are decoder-only.
- Encoder-decoder (the original 2017 Transformer, and T5) runs an encoder over the input and a decoder that attends to both its own output and the encoder's, the natural fit for sequence-to-sequence tasks like translation. Worth naming so the taxonomy is not mistaken for a binary.
The quadratic cost, in words
Because every token attends to every other token, doubling the sequence length roughly quadruples the attention work. A 2,000-token prompt is not twice the cost of a 1,000-token one; it is closer to four times for the attention step. This single fact explains why long context is expensive, why people obsess over the KV cache to avoid recomputing attention during generation, and why "just use a bigger context window" is never free.
Why interviewers probe this
They are checking whether you can explain a transformer without a whiteboard full of matrices, because as an FDE you will explain it to a non-technical buyer. The good answer is "tokens become vectors, attention lets each one look at all the others in parallel, you stack that many times, and the all-pairs attention is why long inputs cost more." The follow-up they hold back is "why is this faster to train than an RNN?" Answer: parallelism. The whole sequence goes through at once instead of one step at a time.
Common misconceptions
- "Transformers process text sequentially like RNNs." They process the whole sequence in parallel; generation is sequential, but the architecture is not.
- "Attention knows word order on its own." It does not. Position information must be injected through positional embeddings.
- "More layers always mean more understanding." Depth helps up to a point, but capability comes from the combination of scale, data, and training, not layer count alone.
- "Encoder and decoder are interchangeable." They differ in what each token can see; decoder-only masks the future, which is what enables generation.
Key takeaways
- A transformer embeds tokens, then uses self-attention so every token draws on every other, stacked across many blocks with residuals and normalization.
- It beat RNNs mainly through parallelism: the whole sequence is processed at once, making training far more efficient.
- Most chat LLMs are decoder-only, meaning each token attends only to earlier tokens, which is what enables left-to-right generation.
- All-pairs attention makes cost grow with the square of sequence length, the root reason long context is expensive.