GO TO A SECTION

↑ ↓ to navigate · Enter to openExplore all sections →
FDEInterviews logo
← Machine Learning & Data Science / 38
hardScaleGoogleDatabricks

The model aced every offline eval and the customer says it's useless. The labels look fine. What now?

Not leakage, not skew, not drift: the model learned exactly what you asked, and you asked for the wrong thing. The proxy-label mismatch that passes every test, the audit that catches it, and why this is the failure no metric can see.

Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.

Not leakage, not skew, not drift: the model learned exactly what you asked, and you asked for the wrong thing. The proxy-label mismatch that passes every test, the audit that catches it, and why this is the failure no metric can see.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 548 remaining answers · ₹2,000 / $25

Every answer, concept and course, every paper breakdown, all published video lessons, hands-on FDE Lab missions, Premium PDF guides and companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.

6 months · One payment · No auto-renewal

Study alongside free video lessons.

Keep practising

Next in this trackThe customer wants to test five model variants without losing money on the bad ones. A/B test or a bandit?PremiumNext in this trackWalk me through SGD vs mini-batch vs Adam, learning-rate schedules, and what vanishing or exploding gradients look like in practice.PremiumNext in this trackExplain backpropagation and why deep networks can be trained at all, without writing a wall of calculus.Premium
FEDITOR'S NOTE

This is the failure mode that survives leakage checks, skew replay, and drift monitoring, because the pipeline is correct and the label is the bug. The screen is whether the candidate audits the label definition against the business outcome rather than re-checking the model, and the reserved probe is label noise versus label bias: random noise caps accuracy, but a systematic proxy mismatch makes a high-scoring model actively wrong, which is far more dangerous and far harder to see.

DISCUSSION · 0

No comments yet — be the first to share your approach.