How do you monitor a model in production when ground-truth labels arrive late, or never?
The senior-MLOps question that exposes dashboard-driven candidates: your accuracy chart is blind for 60 days, or forever. The layered substitute, proxy metrics, score-distribution sentinels, delayed-label backfill, and paid-for labels, and which layer to trust when they disagree.
Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.
The senior-MLOps question that exposes dashboard-driven candidates: your accuracy chart is blind for 60 days, or forever. The layered substitute, proxy metrics, score-distribution sentinels, delayed-label backfill, and paid-for labels, and which layer to trust when they disagree.
Lead with where the obvious approach breaks, because that is the judgment they are screening for — most candidates jump straight to the happy path and lose the room.
Then walk the failure back through the pipeline in order, naming the one metric the customer's exec sponsor actually cares about before you propose the fix.