regression

Model migrations break things that benchmarks never show, format quirks, refusal shifts, tool-calling drift. The eval-replay and shadow-traffic playbook for migrating without surprises.

Provider model upgrades ship faster than your release cycle and quietly shift behavior. The design that turns 'we hope it's fine' into a blocking gate: a frozen golden set, pairwise diffing, segmented scoring, and a shadow-traffic canary.

The regression with no smoking gun, no deploy to blame, no single bad night. Staff-level triage is about reading the trend, not the last run, and the four slow-burn causes that a one-night comparison can't see.

Watch time is the label everyone optimizes and almost nobody measures cleanly. You only observe minutes for videos people chose to play, the distribution is brutally skewed, and the position they saw it in moved the number. The interview is whether you can predict a biased label honestly.