class imbalance

A 0.95 AUC model can still flood a review queue with garbage. Knowing when ROC AUC flatters a model, and what to report instead, separates FDEs who've shipped from those who've only run notebooks.

The accuracy trap is the easy part. Interviewers keep pushing: resample or reweight? Does SMOTE survive contact with production? What happens to your probabilities? Here's the full chain.

A confusion matrix is the 2x2 of right and wrong predictions per class, and it is the reason a 99% accurate fraud model can be useless. The worked imbalance example, and how reading the matrix as precision and recall fixes the lie.

The positive class is maybe 1 in 1,000, your labels arrive late and noisy, and the moment you ship a model the adversary probes it. This is the rare ML problem where the data is actively trying to beat you, so the design is as much about labels, latency, and enforcement cost as about the classifier.

Fraud is well under 1% of transactions, the labels arrive weeks late as chargebacks, and the fraudsters adapt to your model on purpose. Optimizing accuracy gives you a model that approves everything. The interview is dollars-at-risk thresholding, label delay, and a review queue with a fixed headcount.