Explain FlashAttention: why it never writes the full N x N matrix, and how it speeds up attention without changing the math.

The screen is whether the candidate knows attention is memory-bandwidth bound (HBM round-trips on the N x N matrix), not compute bound, and that FlashAttention is an IO-aware tiling that keeps blocks in SRAM and uses online softmax to avoid materializing the full matrix. The reserved follow-up: 'how does softmax work without seeing the whole row, and how does the backward pass avoid storing the matrix.' Saying 'it approximates attention' fails hard; the output is numerically identical (same math, fewer memory round-trips). Bonus for the memory-complexity drop from O(N^2) to O(N) and recomputation in the backward pass.