GO TO A SECTION

↑ ↓ to navigate · Enter to openExplore all sections →
FDEInterviews logo
← Coding & DSA / 65
mediumOpenAIAnthropicHugging Face

Compute cross-entropy loss manually from logits and a target index, no framework.

The loss every language model trains on, built from logits with no torch in sight. The screen for whether you understand log-softmax and the numerical-stability trick, not just import nn.CrossEntropyLoss.

Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.

The loss every language model trains on, built from logits with no torch in sight. The screen for whether you understand log-softmax and the numerical-stability trick, not just import nn.CrossEntropyLoss.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 548 remaining answers · ₹2,000 / $25

Every answer, concept and course, every paper breakdown, all published video lessons, hands-on FDE Lab missions, Premium PDF guides and companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.

6 months · One payment · No auto-renewal

Study alongside free video lessons.

Keep practising

FEDITOR'S NOTE

The single discriminator is the log-sum-exp trick. A candidate who writes log(sum(exp(logits))) without subtracting the max has shipped an overflow that NaNs the moment logits get large, which they do during training. The second tell is computing softmax then taking its log separately (two passes, loses precision) instead of log-softmax directly. The follow-up linking cross-entropy to perplexity is where you separate people who debugged a training run from people who memorized a formula.

DISCUSSION · 0

No comments yet — be the first to share your approach.