21The customer can't randomize, every user must get the new model. How do you measure whether it worked?▼hardGoogleDatabricksMicrosoft1 replies◆ premiumEnterprise reality: legal, fairness or ops constraints kill the A/B test, but the exec still demands proof of impact. The quasi-experimental toolkit, and the honest caveats, that let you answer anyway.Open full answer →
37The customer's marketplace has network effects, so a user-level A/B test is biased. How do you measure the model's impact?▼hardMetaGoogleDatabricks1 replies◆ premiumWhen treatment leaks between units, a clean A/B lies in both directions. The interference taxonomy, when to reach for switchback vs cluster vs geo designs, and the analysis trap that makes naive standard errors useless.Open full answer →