Nothing alerts when quality drops
Meridian's claims assistant has been quietly wrong for three days. Latency is fine, errors are zero, and the evaluation on an unchanged manifest fell from 9 to 6. Find what changed, contain it, and add the alert that should have fired.
- advanced
- about 35 min
- 4 objectives
- par 11 steps
- Meridian Health
This mission is part of premium, with every challenge and incident in the lab. The brief below is the whole problem; the box, the hints and the debrief are what you unlock.
Every answer, concept and course, every paper breakdown, all published video lessons, hands-on FDE Lab missions, Premium PDF guides and companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.
6 months · One payment · No auto-renewal
Study alongside free video lessons.
- Customer
- Meridian Health (synthetic customer; every evaluation, metric and log in this incident is made up for the exercise)
- Situation
- Wednesday 11:00. A claims supervisor reports that the assistant has been recommending the wrong coverage tier since 2026-04-15. Every dashboard is green: p95 normal, error rate zero, no faults. The evaluation runner, which runs nightly on the same eleven cases, shows success fell from 9 to 6 on Monday night and nobody was paged because nothing watches it.
- Your role
- You are the FDE on call for the Meridian deployment. You have the console, the evaluation runner, the provider config (bound to the live router) and the alerting config. You may pin, route or roll back; you may not edit the evaluation cases.
- Goal
- Find what changed under an unchanged manifest, restore the evaluated behaviour on the live system, add the alert that watches quality, and write the status update.
- 1.Restore the evaluated behaviour on the live system
- 2.Commit to the root cause (you commit to an answer)
- 3.Draft an eval-success alert for the same manifest
- 4.Write the status update