GO TO A SECTION

↑ ↓ to navigate · Enter to openExplore all sections →
FDEInterviews logo
← Coding & DSA / 70
hardOpenAIAnthropicGoogle

Implement scaled dot-product attention (the forward pass).

softmax(QKᵀ/√d_k)·V in four lines, but the grade is the three details people drop: the √d_k scale, subtracting the row max for numerical stability, and getting the causal mask to add -inf before the softmax, not after.

Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.

softmax(QKᵀ/√d_k)·V in four lines, but the grade is the three details people drop: the √d_k scale, subtracting the row max for numerical stability, and getting the causal mask to add -inf before the softmax, not after.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 559 remaining answers · ₹2,000 / $25

Every answer, concept and course, every paper breakdown, all published video lessons, hands-on FDE Lab missions, Premium PDF guides and companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.

6 months · One payment · No auto-renewal

Study alongside free video lessons.

Keep practising

Next in this trackCompute the dot product of two sparse vectors.PremiumNext in this trackImplement linear regression with gradient descent from scratch.PremiumNext in this trackCompute the edit distance (Levenshtein) between two strings.Premium
FEDITOR'S NOTE

This is the canonical LLM-team coding screen. Interviewers want to see you reason about tensor shapes out loud, justify the √d_k scale (it keeps softmax out of the saturated regime), and apply the max-subtraction trick so exp does not overflow. The causal mask is the trap: it must set masked logits to -inf BEFORE softmax so they get zero weight, and candidates who zero the weights after softmax break the normalization.

DISCUSSION · 0

No comments yet — be the first to share your approach.