Incident · advanced · 40 min · Premium
The first 48 hours
Halvard's billing rejected every invoice since two in the morning and the carrier has been redelivering ever since. Contain the storm, find the rotated token, restore invoicing, and hand over evidence the next shift can act on.
Brief
- Customer
- Halvard Freight (synthetic customer; every log, token and delivery in this incident is made up for the exercise)
- Situation
- 08:05. Pager:
billing 401 rate 98% since 02:10; receiver retry queue 300 and climbing; carrier redelivering. The night shift restarted the receiver twice. Nothing invoiced for six hours; the carrier's retry backoff means every one of last night's deliveries will come back three more times today. - Your role
- You are the FDE on the Halvard engagement, first day of a two-day incident. You own the receiver's config (bound to the live receiver), the console and the handover packet. Billing is another team's service and they rotated something last night.
- Goal
- Stop the redeliveries from creating duplicates, find why billing rejects the receiver, restore invoicing, drain the retry queue once, and write a handover with evidence for the next 24 hours.
Objectives
- 1.Stop late acks from driving redeliveries while billing is being fixed
- 2.Restore invoicing: billing accepting the receiver again
- 3.Drain the retry queue exactly once, including dead letters
- 4.Commit to the root cause (you commit to an answer)
- 5.Write the handover with evidence for the next 24 hours
Mission workspace
This mission is part of premium, with every challenge and incident in the lab. The brief above is the whole problem; the box, the hints and the debrief are what you unlock.
Get full Premium access · ₹2,000 / $25
Every answer, concept and course, plus Premium PDF guides, companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.
6 months · One payment · No auto-renewal
Study alongside free video lessons.