MLOps & ML Engineering: the practice test

A service's behavior is its code, so testing code is testing the service. A model's behavior comes from code, data, and the training run, which means reproducing it needs all three pinned, and validating it needs statistical checks (accuracy on a holdout, slice metrics, calibration) alongside unit tests. The consequence that surprises DevOps teams: a model pipeline can go red with zero commits, because the world's data drifted. Everything else in MLOps, registries, drift monitors, retraining triggers, follows from that one asymmetry.

Data drift is the inputs changing; concept drift is the world changing its mind about what the inputs mean. Fraudsters adapting tactics is the canonical case: the same-looking transactions now carry different fraud probability, so a model trained on the old relationship degrades while every input-distribution monitor stays green. That is what makes concept drift the more dangerous of the two: it is invisible without outcome feedback, which arrives late. The practical countermeasures follow: monitor performance on labeled samples where possible, and treat stable-inputs-falling-precision as a signature worth alerting on.