Core

Numerical Stability

When you implement softmax, cross-entropy, or a running average by hand, the naive formula overflows or loses precision on real inputs. The fixes are a small, reusable toolkit: subtract the max before exponentiating, work in log space with log-sum-exp, and accumulate carefully. ML-adjacent coding screens lean on this, because the candidate who writes exp of a large logit and gets inf has shipped a silent bug.

a free account unlocks the core curriculum tier · no card

RELATED CONCEPTS

Temperature, Top-p and Sampling Information Theory for ML: Entropy, Cross-Entropy, KL and Perplexity Precision, Recall and F1

PRACTICE THIS IN REAL QUESTIONS

LLM & GenAI FundamentalsExplain FlashAttention: why it never writes the full N x N matrix, and how it speeds up attention without changing the math.→Coding & DSAImplement top-p (nucleus) sampling from a list of logits in pure Python.→Coding & DSACompute cross-entropy loss manually from logits and a target index, no framework.→LLM & GenAI FundamentalsSpend a 1000-token test-time budget on a math problem: process-reward scoring with tree search.→ML Infrastructure & GPUsCapacity-plan 10B 768-dim embeddings: HNSW vs IVF-PQ storage, index, and RAM for sub-100ms.→Coding & DSAImplement scaled dot-product attention (the forward pass).→