training
FDE interview questions tagged training, across every topic.
3 questions · 0 unlocked for you
Concepts behind "training"
The curriculum that explains the ideas these questions test.
Foundational
Gradient Descent & Learning RateGradient descent is how almost every model learns: compute the slope of the loss with respect to the weights, then step the weights a little in the downhill direction. The learning rate sets the step size, and it is the single most consequential knob. Too small and training crawls; too large and it overshoots and diverges.📊 Evaluation & ML Foundations
Foundational
Loss FunctionsThe loss function is the objective you actually optimize, and choosing it wrong quietly sabotages everything downstream. MSE punishes outliers, MAE ignores their size, Huber splits the difference, cross-entropy is the default for classification, and contrastive losses shape embeddings. The rule: the loss must match the metric you are judged on.📊 Evaluation & ML Foundations
Core
Knowledge DistillationDistillation trains a small student model to mimic a large teacher, learning from the teacher's full output distribution rather than just hard labels. The soft targets carry extra signal about how the teacher 'thinks', so the student keeps much of the quality at a fraction of the size and latency. Knowing when distillation beats quantization or pruning is standard FDE ground when you have a latency or cost budget to hit.🖥️ ML Infrastructure & ServingSign in
Core
Normalization: Batch vs LayerNormalizing activations keeps their scale stable as they flow through a deep net, which speeds training and lets you use higher learning rates. Batch norm normalizes across the batch and has nasty train/eval and small-batch pitfalls; layer norm normalizes across features per example, which is why transformers use it. RMSNorm strips it down further.📊 Evaluation & ML FoundationsSign in