Guided · advanced · 35 min · Premium

Kill the worker

Calder's action worker sent a replacement request and died before recording the receipt. Reproduce the crash, find out what the service actually committed, choose a recovery that cannot ship a second pump, and prove it.

Brief

Customer: Calder Equipment (synthetic customer from the Production AI Agents course; every ledger, receipt and log on this box is made up for the exercise)
Situation: At 03:12 the action worker on calder-agent-01 was killed by the host during a memory-pressure event. Its ledger for case C01 says intent: sent, receipt: null. The replacement service may or may not have accepted the request. The worker's current recovery mode is resend. An operator is asking whether a pump for C01 is on its way, and whether restarting the worker will send another.
Your role: You are the FDE who owns the worker. You may read the ledger and logs, query the replacement service (it is a simulator with the production contract: idempotency key tenant:case, digest of the proposal, receipts), run the worker in recorded mode, and edit worker.yaml. Anything the worker sends is treated as sent.
Goal: Reproduce the crash on a fresh ledger, establish from the service what was committed, choose the recovery policy that reconciles from the service's receipt rather than guessing, configure it, and recover C01 to a ledger that matches the service. Ship nothing twice.

Objectives

1.Reproduce the crash: send C01 and kill the worker before the receipt is recorded
2.Establish from the service what was committed for C01
3.Choose the recovery policy for an unknown outcome (you commit to an answer)
4.Configure the worker to reconcile from the service, keeping one key per logical action
5.Recover C01 so the ledger carries the service's receipt, with no duplicate request created on this box

You will practise

→Explain the partial failure between sending an action and recording its outcome
→Recover an unknown outcome by asking the service, never the model or the local ledger
→Choose an idempotency key per logical action and say why a per-attempt key resends
→Reconcile a ledger to the service's receipt and prove no duplicate effect occurred

The mechanism, explained

Course lesson: persist intent before sending
why the ledger says sent with no receipt, and why that is the correct thing for it to say
Course lesson: recover an unknown outcome
the recovery procedure this mission makes you run against the real contract
Agent fleet failure recovery with executed side effects
the interview question: what to do when the effect may already have happened
Idempotency
same key, same effect: the service-side property that makes reconciliation possible
Agent side effects, idempotency and compensation
when a duplicate cannot be prevented and must be compensated instead

Do first

The webhook that fires twice

Mission workspace

This mission is part of premium, with every challenge and incident in the lab. The brief above is the whole problem; the box, the hints and the debrief are what you unlock.

Get full Premium access · ₹2,000 / $25

Every answer, concept and course, plus Premium PDF guides, companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.

6 months · One payment · No auto-renewal

Study alongside free video lessons.

← all missions next: Pause writes, keep lookups →