tokenization

Modern LLMs share one architectural skeleton. Learn the five components and you can read almost any model card cold, plus the single distinction that proves to an interviewer you actually understand it rather than the buzzwords.

Tokens drive your customer's bill, their latency, and the weird failures like botched arithmetic. The rough numbers every FDE should rattle off, plus the trap answers interviewers listen for.

BPE is how GPT and most LLMs turn text into tokens, and building a tiny trainable version from a corpus is a favorite deep-dive. The screen for whether you understand the merge loop, deterministic tie-breaking, and round-trip decode, not just tiktoken.encode.