numerical stability

The loss every language model trains on, built from logits with no torch in sight. The screen for whether you understand log-softmax and the numerical-stability trick, not just import nn.CrossEntropyLoss.

softmax(QKᵀ/√d_k)·V in four lines, but the grade is the three details people drop: the √d_k scale, subtracting the row max for numerical stability, and getting the causal mask to add -inf before the softmax, not after.