GO TO A SECTION

↑ ↓ to navigate · Enter to openExplore all sections →
FDEInterviews logo
← LLM & GenAI Fundamentals / 46
easyOpenAIGoogleAnthropic

Decoder-only, encoder-only, encoder-decoder: what is each good at, and why does generation use decoder-only?

Three transformer families, one distinction that decides everything: which tokens are allowed to see which. The attention mask is the whole answer, and it explains why GPT generates and BERT does not.

Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.

Three transformer families, one distinction that decides everything: which tokens are allowed to see which. The attention mask is the whole answer, and it explains why GPT generates and BERT does not.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 548 remaining answers · ₹2,000 / $25

Every answer, concept and course, every paper breakdown, all published video lessons, hands-on FDE Lab missions, Premium PDF guides and companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.

6 months · One payment · No auto-renewal

Study alongside free video lessons.

Keep practising

Next in this trackWhat is RoPE, why does relative position help, and why does it extend to long context better than learned absolute embeddings?PremiumNext in this trackPre-norm vs post-norm transformers: why do large models use pre-norm, and how does it affect training stability?PremiumNext in this trackExplain FlashAttention: why it never writes the full N x N matrix, and how it speeds up attention without changing the math.Premium
DISCUSSION · 0

No comments yet — be the first to share your approach.