Course outline · choose a lesson
Test the stated request against the work people actually do
Use observation, interviews and a small timing packet to separate a requested feature from an evidenced workflow problem. Record assumptions, quantify active work carefully and choose a next test without treating one observation as a universal diagnosis.
28 MIN
By FDEInterviews · Updated
Transcript
We want a chatbot over our documentation. That request already proposes an interface, a source and a way of interacting. Treat it as a hypothesis. It may address a real search problem, or it may miss an earlier identification step or a later approval delay. Neither accepting it nor rejecting it on its wording tells you the right solution.
Four questions. Walk me through the last specific instance. How long does each step take, and how often? Separate active effort from waiting. What happens if nothing changes? It may be delay, errors or staff burden, not an immediate cash cost. And what evidence would change your view? That invites the case that disproves you.
Then watch the work in its real setting, alongside interviews and records, so each catches what the others miss. Agree what may be observed or recorded, and don't interrupt a critical task to ask a question. Ask neutrally. What tells you these IDs refer to the same customer? is far more useful than, why do you still fix this manually? Workflow research isn't an evaluation of the person doing it.
Here's a synthetic observation note. The analyst says the morning report takes twenty minutes, most days. We watched from eight fifty to nine thirty. Among eleven steps, the analyst searches the CRM to decide whether two IDs are one customer, fixes them from a personal mapping file, overwrites the report's totals, and deletes test orders recognized by their delivery address. The 40 minutes on the clock don't prove 40 minutes of effort, because questions, waiting and atypical work may account for some of the difference, and the note has no per-step timing at all.
Keep observation, inference and the next test in separate columns. The analyst checks doubled names in the CRM. That suggests names alone aren't enough, so get the canonical identity rules and collision cases. A personal mapping file supplies corrections. That suggests a recurring source mismatch, so get its history and owner. Test orders are spotted by address, so the classification may be a guess that could remove legitimate orders, and the next test is a supported marker, plus the false positives it would have caught.
So a good problem statement names the task, the people, the friction and the uncertainty. In the observed morning report, the analyst reconciled customer identifiers by hand before publishing totals. We need to measure that step and verify the mapping rule. Document search remains a separate question. It doesn't prescribe a chatbot, or a new master-data system, yet.
Now a separate synthetic timing fixture: four report batches took 4, 6, 8 and 22 minutes. That's 40 minutes in total, a mean of 10, and a median of 7. Here's the trap. 7 times four batches is 28, but the batches actually took 40, because that single 22-minute case carries more than half of it. So use the total or the mean for how much work there is, and the median for what a typical batch feels like.
So treat the requested feature as a hypothesis, not a mistake. Combine specific examples, observation and records, keep observations apart from causes and fixes, and leave with a problem statement you can test. The full lesson is on fdeinterviews.com.
TL;DR: Treat a requested feature as a hypothesis about a need, then compare it with specific work, evidence and constraints. Combine observation with questions and measurement before deciding what to build, change or leave alone.
Where you are. The role module established the engagement loop. Discovery now turns a broad request into a problem statement that can support a bounded scope and an honest evaluation.
A request may contain an answer worth testing
“We want a chatbot over our documentation” proposes an interface, a source and an interaction pattern. It may address a real search problem, or it may miss an earlier identification step or a later approval delay, and neither accepting nor rejecting it on the wording alone tells you the right solution.
Ask what happened the last time someone needed the information: who was doing what, which record or decision was missing, what they tried and what happened next. Preserve the person's explanation rather than translating it immediately into your preferred architecture.
A useful discovery result can confirm the requested feature, or it can point to a smaller workflow change, a source repair or a decision not to proceed. Do not assume every customer has misunderstood their own problem or that an expensive step must be obvious after one short visit.
Questions that reveal evidence
Walk me through the last specific instance. A concrete case exposes details a general description tends to leave out. Follow up on unusual steps, but also ask whether this case was typical and what differs across teams, channels or shifts.
How long does each step take, and how often? Distinguish active effort from elapsed waiting, one case from a batch, and an estimate from a measured value. Frequency times an appropriate mean can estimate total effort under explicit assumptions. Multiplying a median by volume or overlapping work by several users can misstate the total.
What happens if nothing changes? The consequence may be labor, delay, errors, missed obligations, inaccessible service or staff burden. A problem need not show up as immediate cash spend to matter, so ask how it competes with other priorities and who can decide, rather than dismissing it as merely annoying.
What evidence would change your view? This invites the user and team to name a disconfirming case. If a candidate identifier repair is correct and the task still takes just as long, some other step dominates, which is why you plan to observe the whole task after the intervention.
Observation with a clear purpose
GOV.UK's contextual-research guidance describes observing activities in their everyday setting to understand needs, barriers and workarounds. For this course, use observation alongside interviews and records so each can expose what the others miss.
Agree what may be observed or recorded, and protect confidential information. Avoid interrupting a critical task with a question; note it and ask at a suitable pause. The observer's presence and an unusual day's workload can change behavior, so record context and seek additional cases where needed.
Someone may skip mentioning a routine step because it feels obvious, or deliberately take a shortcut for a reason the observer has not yet understood. Ask neutrally: “What tells you these IDs refer to the same customer?” is more useful than “Why do you still fix this manually?” Do not turn workflow research into an evaluation of the employee.
The supplied observation, before any fix
This fictional note preserves a detailed workflow; the quotes are teaching material, not statements from real customers.
Discovery notes, day two: the request at the top, the workflow underneath
The stated request (the director, kickoff call): "We want a chatbot over our documentation. People can never find anything."
What the analyst said when asked how the morning report works: "I pull the report and check it. Twenty minutes, most days."
What we watched, 08:50 to 09:30:
- Opens the reporting tool, runs Daily Orders, exports it to a spreadsheet
- Opens yesterday's spreadsheet beside it and copies her own formula columns across
- Sorts by customer name and scans for names that appear twice
- For each doubled name, opens the CRM in a second tab and searches to establish whether the two identifiers are the same customer [1]
- Fixes the identifiers by hand, from a mapping she keeps in a personal file called fixes-do-not-delete.xlsx [2]
- Re-totals the affected customers and overwrites the report's numbers
- Spot-checks the three largest accounts against the billing system
- Deletes the overnight test orders, which she recognizes by their delivery address
- Pastes the totals into the team's running workbook
- Writes two lines of commentary
- Sends to the distribution list at 09:28
Steps 4 to 6 suggest identifier reconciliation is a candidate problem. The observation does not establish its typical duration, correctness or whether document search is also a problem.
- [1] Observed reconciliation motivates a mapping investigation. Check the source rules rather than assuming duplicate names mean the same customer.
- [2] The personal file is a dependency to understand. Verify its owner, provenance and changes before treating its mappings as authoritative.
The session runs from 08:50 to 09:30, and the report is sent at 09:28. That does not prove 40 minutes of active reporting effort or contradict the analyst's “twenty minutes, most days.” Questions, waiting and atypical work may account for some of the difference, and the note has neither per-step timing nor a representative sample.
Other risks are visible too. Recognizing test orders by delivery address may remove legitimate orders, and manual overwriting may lose provenance, the record of where a number came from and what changed it. The mapping file may contain valid domain knowledge or stale corrections. Treat each as a question with an owner and test, not as an automatic target for deletion.
Observation, inference and next test, kept apart
| Observation from the note | Candidate inference | Next evidence |
|---|---|---|
| Analyst checks doubled names in CRM | Names alone are insufficient identifiers | Canonical identity rules and collision cases |
| Personal mapping file supplies corrections | A recurring source mismatch may exist | Mapping history, owner and source reconciliation |
| Report totals are overwritten | Output provenance may be weak | Original values, change record and reconciliation process |
| Test orders are recognized by address | Test/live classification may be heuristic | Supported test marker and false-positive examples |
| Director asks for document search | Search may be a separate unmet need | Recent failed searches and their workflow consequences |
A good problem statement names the task, affected users, observed friction and uncertainty: “In the observed morning report, the analyst reconciled customer identifiers manually before publishing totals. We need to measure that step and verify the mapping rule; document-search demand remains a separate question.” It does not yet prescribe a chatbot or a new master-data system.
A modest timing estimate
The following separate synthetic timing fixture contains active reconciliation minutes for four observed daily report batches. It is not the timing detail omitted from the narrative note.
from statistics import mean, median
minutes = [4, 6, 8, 22]
print('observed total:', sum(minutes))
print('mean per batch:', mean(minutes))
print('median per batch:', median(minutes))
assumed_monthly_batches = 20
estimated_hours = mean(minutes) * assumed_monthly_batches / 60
print('projected monthly hours:', round(estimated_hours, 2))
assert sum(minutes) == 40 and mean(minutes) == 10 and median(minutes) == 7
# observed total: 40
# mean per batch: 10
# median per batch: 7.0
# projected monthly hours: 3.33
The observed total is 40 minutes, and the 3.33-hour projection assumes these four batches represent twenty future batches; it is not measured monthly effort or proven avoidable work. The fixture also shows the median trap in miniature. The median is 7 minutes, and 7 times four batches is 28, but the four batches actually took 40, because the one 22-minute case carries more than half of the total. A median hides the tail; a total counts it. Use the total, or the mean, when the question is how much work there is, and the median when the question is what a typical batch feels like. A fix may leave review effort in place, add maintenance or shift work to someone else, so estimate the intervention's effect separately.
If the 22-minute case reflects a valid month-end condition, deleting it because it is inconvenient changes the estimate. If a second analyst works simultaneously, distinguish labor minutes across both people from elapsed task time. Later measurement lessons explore these distinctions in more depth.
The spine above is the whole move in order, from a request that already contains a guessed answer to a statement you can test, with the three-column split marked as the step that keeps a guess from hardening.
Do this before moving on
From the observation note, write five observations and keep interpretations in a separate column. Choose one candidate problem and specify a next test that could disprove it. Include a case where the chatbot request remains appropriate.
Run the timing fixture and change the last value to 10: the total becomes 28, the mean seven and the median seven. The same median now sits beside a different total. Explain why the sample and a volume assumption belong beside any projected effort figure.
Finally, observe or reconstruct a familiar task under appropriate permissions. Describe the complete path, including waiting, checks and exceptions, and do not rank steps by count alone: three quick corrections may matter less than one long approval delay. Your result should support a testable problem statement without pretending the next design decision is already settled.
Go deeper
- Requirements discovery covers the interview technique in more depth, including how to avoid leading the answer.
- Scoping ambiguous problems is the stage immediately after this one, and it fails when discovery was shallow.
- Stakeholder management matters because the person who described the problem is rarely the person who approves the fix.
- Data quality names the upstream breakage that most compensating workarounds are hiding.
- The most ambiguous project you have run end to end is where interviews ask you to demonstrate exactly this.
Key takeaways
- A requested feature is a hypothesis to investigate, not automatically a mistake.
- Combine specific examples, observation, interviews and records.
- Keep observations separate from inferred causes and proposed solutions.
- Distinguish active effort, elapsed time, batch volume and sampling assumptions.
- Leave discovery with a testable problem statement and visible unresolved questions.
Check yourself
Answer before you look. Recalling it is what makes it stick; recognising it does not.
1The director asks for document search, while one observed report needs manual ID reconciliation. What follows?
2The timing fixture has values 4, 6, 8 and 22. Which supports the observed total?
3Why does watching one 40-minute session not prove the task normally takes 40 active minutes?
Sign in free to save your progress and return to this lesson.