Challenge · intermediate · 25 min · Premium
Golden set drift
Meridian's evaluation passes every night and the assistant is wrong about the new coverage rules: the golden set was written under last year's policy. Find the labels that rotted, version the manifest, and make the run tell the truth.
Brief
- Customer
- Meridian Health (synthetic customer; every case, label and policy on this box is made up for the exercise)
- Situation
- Coverage policy CP-7 changed on April 1: dental now requires a referral. The nightly evaluation still passes 11 of 11. Supervisors report the assistant tells members no referral is needed. The golden set's expected answers say the same thing the assistant does, because they were written in January.
- Your role
- You are the FDE who owns the evaluation set. You have the golden set, the policy history, the manifest, and the runner. You may edit expected answers only with the policy reference that justifies each change.
- Goal
- Find the cases whose expected answers contradict the current policy, fix them with a reference, version the manifest so the run names the policy it was scored against, and show the run failing honestly.
Objectives
- 1.Count the golden cases whose expected answer contradicts the current policy (you commit to an answer)
- 2.Name why a passing evaluation missed it (you commit to an answer)
- 3.Correct the stale expected answers with a reference, and version the manifest
- 4.Show the run failing honestly against the named policy
Mission workspace
This mission is part of premium, with every challenge and incident in the lab. The brief above is the whole problem; the box, the hints and the debrief are what you unlock.
Get full Premium access · ₹2,000 / $25
Every answer, concept and course, plus Premium PDF guides, companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.
6 months · One payment · No auto-renewal
Study alongside free video lessons.