hardMetaGoogleDatabricks

Offline metrics improved, but the online A/B shows no business lift. How do you explain the gap, and what do you do?

The maturity exam of production ML: a model that's better on paper and worthless in production. Five distinct causes, the funnel diagnostic that separates them, and when the right call is don't ship.

Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.

The maturity exam of production ML: a model that's better on paper and worthless in production. Five distinct causes, the funnel diagnostic that separates them, and when the right call is don't ship.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 548 remaining answers · ₹2,000 / $25

Every answer, concept and course, every paper breakdown, all published video lessons, hands-on FDE Lab missions, Premium PDF guides and companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.

6 months · One payment · No auto-renewal

Study alongside free video lessons.

Keep practising

Next in this trackDesign the path from a notebook pilot to a production ML system at a customer, end to end.Premium Next in this trackEstimate the number of gas stations in the United States, then tell me how you'd verify it with data.Premium Next in this trackWhat are the assumptions of linear regression, and how does a random forest actually work?Premium

DISCUSSION · 0

No comments yet — be the first to share your approach.

Offline metrics improved, but the online A/B shows no business lift. How do you explain the gap, and what do you do?

Read the related research

Keep practising