Course outline · choose a lesson
You have two days, no context, and everybody has a different theory
Build a bounded rescue assessment from a supplied freight-deployment evidence packet. Separate containment from diagnosis, test competing explanations, distinguish stakeholder concerns from technical findings, and make the next commitment depend on evidence and access.
30 MIN
By FDEInterviews · Updated
Transcript
You're joining an engagement that already has software, history and a disappointed customer. In the page's fictional freight case, a five-month-old deployment hasn't reached its intended scope, operators still resolve many exceptions by hand, and the sponsor wants a decision about continuing. Those facts describe a situation. They don't establish the cause, and your job isn't to prove the previous team wrong.
Look at four things, each with its own evidence. System behavior. Workflow fit. Delivery confidence. And feasibility. And if the sponsor sends a delegate, ask what they can decide. It doesn't prove lost trust.
But if the system is sending wrong notifications, corrupting data or exposing private information, don't wait two days to finish an assessment. Use the authorized incident process to pause the action, route work to a staffed fallback, or apply a verified, reversible mitigation. A rollback can itself be unsafe once state has changed.
Otherwise, treat the two days as a planning example, not a rule. If a needed export arrives late, deliver a partial assessment on the agreed date and say what can't be established yet. Keep a small hypothesis register, and choose each test by how well it separates the plausible explanations at an acceptable risk.
Now a synthetic evidence packet. On day one, 20 of 1,000 inputs failed. On day two, 40 of 2,000. Failures doubled, but the rate stayed at 2%. On day three, 220 of 1,000 failed, 22%, the same day a new source format started. That's a correlation. Two things changed on one day. And application B only arrived on day four, after the rate had already jumped.
E4 is different. The original application accepts a region as a string and rejects the same code nested in an object. It holds everything else constant and changes only the shape of one input, and that's what separates a discriminating test from a coincidence in a timeline. E5 breaks down the 550 failures: 500 recognized codes in the nested shape, 25 unknown codes and 25 bad dates, and E6 says nobody has provided a destination-side action ledger, so replay safety is unknown.
So test three claims. Application B caused the onset? The timeline contradicts that. The parser fix will recover 95.5% of failures? That's 525 of 550, but only 500 are the shape the fix handles, so it's about 90.9%. The model needs replacing? Nothing here establishes that.
The assessment's next step is to replay all 550 failures in an isolated store with external actions denied, and record the result rather than promising it. The commitment is conditional. If the export is late, deliver the checks you've done and the list of what's missing, on the date you agreed.
And sometimes the evidence supports stopping. The use case may lack value, the access may never come, or a safe service may cost more than the scope supports. Say so, with the alternatives you examined. Don't invent a replacement just to make the stop sound positive. Stopping has duties too, from data export to a handover to manual work.
So contain active harm first, keep reproduced mechanisms apart from recovery predictions and unknown effects, and make every commitment conditional and reviewable. The full lesson is on fdeinterviews.com.
TL;DR: Contain active harm when needed, then build an assessment that separates observed facts, tested mechanisms and unresolved explanations. Agree a near-term update you can deliver, and make repair forecasts conditional on evidence, access and the customer's change process.
Where you are. You are joining an engagement that already has software, history and disappointed expectations, and your task is to establish its current state and a feasible next decision, not to prove that the previous team was wrong.
The workflow and the working relationship
In this fictional freight engagement, a five-month-old deployment has not reached its intended operating scope. Operators still resolve many exceptions manually, the original engineer now supports another account, and the sponsor wants a decision about continuing. Those facts describe a situation to investigate; they do not establish the cause or whether the software is salvageable.
Technical performance, workflow fit and confidence in delivery can all need attention, and while they influence one another, each needs its own evidence. A working parser does not establish that users have access, the workflow is useful or the customer accepts the remaining support burden.
| Dimension | Evidence to seek | Avoid inferring |
|---|---|---|
| System behavior | Versioned inputs/outputs, failures, reproduction and current configuration | The loudest complaint must name the mechanism |
| Workflow fit | Work attempted, assistance, unresolved cases and operator constraints | A successful demo means routine work is covered |
| Delivery confidence | Missed/kept commitments, unanswered decisions and stated concerns | A missed meeting or delegated attendee proves loss of trust |
| Feasibility | Data, access, owners, change windows and remaining effort | The problem must be small because a new engineer found it |
Ask the sponsor what decision they need and how progress will be judged, ask operators to show examples of unfinished work, and ask the technical owners which access and recovery controls are available. A delegate may be the right decision-maker; establish their authority instead of diagnosing the relationship from seniority or attendance.
Containment can precede a complete diagnosis
If the system is sending incorrect notifications, corrupting data or exposing private information, waiting two days to finish an assessment can increase harm. Use the authorized incident process to reduce exposure: pause the affected action, route work to a staffed fallback or apply a verified reversible mitigation. Preserve useful evidence when feasible without making evidence collection a prerequisite for urgent containment.
Google's incident-response chapter describes stopping impact before completing root-cause investigation, and it shows why mitigation and understanding the underlying fault are separate tasks. Apply the principle to the actual risk and available controls; a rollback can itself be unsafe when state has changed.
A well-understood fix can be appropriate on arrival, provided you record the trigger, evidence, scope, validation, owner and recovery plan. An impressive-looking patch with no reproduction or acceptance test, on the other hand, is no substitute for diagnosis, and the clock alone decides neither case.
The first two days as an assessment cadence
The two-day outline is a planning example, and access restrictions, incident urgency or source retention can change the order. If a required export will arrive later, deliver a partial assessment on the agreed date and say what cannot yet be established, rather than promising an exact diagnosis date that depends on unavailable evidence.
Speak with the sponsor, operators, customer technical owners and your own delivery team. Separate conversations can make constraints easier to hear and a joint review can resolve contradictory definitions, though neither format guarantees candor. Ask for the workflow, expected behavior, a recent concrete failure and changes around its onset.
Keep a small hypothesis register, and choose the next test by how well it distinguishes plausible explanations at acceptable risk and cost. One active experiment can keep the work focused, while a rule allowing only one hypothesis encourages tunnel vision.
Grading the supplied evidence packet
The following packet shares the D1–D5 freight example used in this module's diagnosis lab. D1–D5 are complete synthetic UTC days, not dates from a real customer incident.
| Evidence | Supplied statement | What it supports |
|---|---|---|
| E1 | D1: 1,000 distinct inputs, 20 failed; D2: 2,000 inputs, 40 failed | Absolute failures doubled while both rates remain 2% |
| E2 | D3: 1,000 inputs, 220 failed; source format S2 begins at D3 | A 22% failure rate coincides with the recorded format change |
| E3 | Application B begins at D4; D3 still used A | B's recorded deployment is later than this observed rate increase |
| E4 | Fixed application A accepts a string region and rejects the same code nested in an object | A reproduced parser-shape mechanism on a paired record |
| E5 | D3–D5 breakdown: 500 nested recognized codes, 25 genuinely unknown codes, 25 bad dates | A conditional 500/550 recovery target for a shape repair |
| E6 | No destination-side action ledger has been provided | Replay safety for external effects remains unestablished |
E2 supplies a correlation: two things changed on the same day, and the packet cannot say which caused which. E4 demonstrates one mechanism, because it holds everything else constant and changes only the shape of one input, which is what separates a discriminating test from a coincidence in a timeline. E5 predicts a bounded recovery opportunity only if the full selected population matches that breakdown and passes replay. None establishes safety to resend every failed-looking record into production.
Practice with three statements:
- “Application B caused the onset.” The supplied timeline contradicts that narrow onset claim, assuming the deployment records are complete and correctly aligned. B could still introduce another problem later.
- “The compatible parser will fix 95.5% of failures.” Unsupported. Unknown-region labels cover 525/550 failures, which is 95.5 percent, but only 500 are the supplied shape-compatible subset: 500 ÷ 550 is about 90.9% of failures.
- “The model needs replacement.” Not established by this packet. Test the parser path and retain any independent model-quality concern as a separate hypothesis.
A confidence label should describe what it rests on. “Reproduced on one paired input; population recovery pending full replay” is more useful than an unexplained “90% confident.” It also tells the next engineer what evidence would strengthen or weaken the claim.
Tests that can change the decision
| Hypothesis | Discriminating evidence | What would change the assessment |
|---|---|---|
| The S2 shape triggers parser rejection | Same logical input and fixed app, change only the region representation | If both shapes pass, investigate the production path/configuration |
| Load explains the failure increase | Arrival bursts, service times and resource saturation on aligned windows | A repeatable load failure not explained by input shape adds another mechanism |
| Model quality caused unresolved cases | Paired reviewed outputs after successful ingestion | Bad answers among correctly parsed cases justify a separate model investigation |
| Failed-looking work already caused external effects | Destination receipts keyed to the selected records/actions | Applied or unknown effects change the replay plan |
Do the safe local replay before changing a production parser, and ask for the most valuable missing evidence with a precise scope and purpose. A request for “all logs from the last six months” can be slow, expensive and unnecessarily sensitive; start with the failing interval and representative identifiers, then expand as justified.
An assessment that travels accurately
Rescue assessment: parser mechanism confirmed, full recovery still conditional
Current impact
D3 failure rate is 22% of distinct received records, compared with 2% on D1 and D2. The supplied D3–D5 set contains 550 failed records out of 2,500 inputs.
Established mechanism
Application A rejects the new nested region representation in a paired local replay. The recorded application B deployment occurs later than the observed onset. [1]
Proposed next step
Confirm the source contract and replay all 550 selected failures in an isolated output store with external actions denied. The supplied breakdown predicts 500 shape-compatible recoveries and 50 residual exceptions. Record the result rather than promising it in advance.
Unresolved boundary
We do not yet have destination action receipts, peak-window behavior or evidence for other source versions. Production replay waits for effect reconciliation and the approved change process. [2]
Next commitment
Deliver the isolated replay report at the agreed checkpoint, conditional on receipt of the selected export. If access is delayed, deliver the completed checks and the missing-evidence list at that checkpoint.
- [1] This distinguishes a reproduced mechanism from a broader causal claim about every failure.
- [2] An unknown external effect is a recovery constraint, not permission to try the action again.
Report any containment and repairs already performed, and keep the business consequence visible: affected work, manual fallback capacity and outstanding decisions. A technically precise memo that never says what operators can do next leaves the rescue incomplete.
Repairing commitments without inventing a relationship diagnosis
Agree the update cadence and audience. Daily notes can help during active recovery, but send useful changes, decisions and blockers rather than making a universal fourteen-day email rule. Record what you promised, what happened and what changed the forecast.
Discuss past defects with evidence and respect, remembering that blameless reporting does not mean concealing a known error or refusing to explain how it affected the customer. Describe contributing conditions, impacts and corrective work without unsupported judgments about motives or competence. Google's postmortem guidance gives a concrete example of that approach.
A small visible improvement is worthwhile when it helps the workflow and fits the risk and effort budget, but it cannot justify deferring a serious integrity problem or pretending that a cosmetic fix establishes recovery. Choose something the operator can verify, such as a reconciled unresolved-work queue with accurate reasons.
Recommending a stop when the evidence supports it
The use case may lack value, required access may be unavailable, or a safe service may cost more than the agreed scope can support. State the constraint, the alternatives examined and the remaining uncertainty. A narrower option is useful when evidence supports it, but do not invent a viable replacement just to make a stop recommendation sound positive.
Stopping can itself require data export, retained evidence, outstanding-effect reconciliation and an agreed transition to manual work, so name those obligations. The decision belongs with the authorized product, customer and commercial owners; your assessment should make it reviewable.
The spine above is the two days as a sequence, with containment sitting before diagnosis rather than after it.
Do this before moving on
Write a one-page assessment from E1–E6 before reading the example memo again, labeling observations, inferences, competing explanations and unknowns. Reproduce the 2%, 22%, 95.5% and 90.9% figures with their denominators, and explain which one is relevant to the parser recovery target.
Now add E7, in which an authorized operator confirms that duplicate notifications are still going out. Add immediate containment and effect reconciliation to the plan rather than waiting for the end-of-day assessment. State what would prove the notifications have stopped and what might still be queued remotely.
Finally, change D2 to 440 failures out of 2,000 inputs, so the 22% rate now precedes the recorded S2 change. Revise the onset claim and request earlier inputs and change evidence. The paired parser defect remains demonstrated, but it no longer explains the whole timing story by itself.
Go deeper
- Diagnosing intermittent timeouts without customer code practices narrow evidence requests across a restricted access boundary.
- Proving what happened builds the record-level timeline and denominator checks behind this assessment.
- Salvaging a pilot that missed its metric connects technical findings to a bounded continuation decision.
- An executive escalation when scope and timeline broke rehearses communicating the impact and required decision without blame.
- Stakeholder management identifies the owners and authority needed to act on findings.
- Orienting in someone else's system supplies a technical exploration method for unfamiliar deployments.
Key takeaways
- Contain active harm when needed; a two-day assessment cadence is not a ban on early verified repairs.
- Separate system behavior, workflow fit and stated delivery concerns instead of inferring trust from attendance.
- Keep competing explanations and use tests that can change the next decision.
- Distinguish reproduced mechanisms from population recovery predictions and unknown external effects.
- Make commitments reviewable, conditional where necessary and explicit about the next owner or decision.
Check yourself
Answer before you look. Recalling it is what makes it stick; recognising it does not.
1Duplicate notifications are still being sent while you prepare the assessment. What takes priority?
2Why is 500/550 the supplied parser recovery target?
3A sponsor sends a delegate. What can you conclude?
4D2 is revised to a 22% failure rate before the recorded S2 rollout. What changes?
Sign in free to save your progress and return to this lesson.