data leakage

This exact gap kills more customer pilots than any modeling choice. There's a ranked list of culprits, and one of them hides inside innocent-looking feature pipelines at almost every enterprise.

The most expensive leakage bug in feature-store ML: training joins that grab feature values from after the label event. The timeline that exposes it, the one query that proves it, and the fix that survives audit.

The validation set is the one you tune and select on; the test set is the one you only look at once. Confusing them, or tuning so hard you overfit the validation set, is how a model that aced offline metrics dies in production.

A model that's too good is a bug, not a win. The screen is whether your instinct on a suspiciously high number is to celebrate or to get suspicious, and whether you disclosed the leak instead of quietly fixing it.