TL;DR: An agent is the wrong answer whenever the steps are predictable: climb the ladder from single call to fixed workflow to agent loop only when the task has unpredictable structure, the model has tool feedback to check itself, and autonomy is worth the compounding failure cost.
How to approach it
This question is a seniority test disguised as a definition. The interviewer is checking for simplicity bias: candidates burned by production agents argue against them by default. Open by defining the spectrum, then make the case that most enterprise problems live at the cheap end.
A strong answer
Define the ladder: (1) a single LLM call with a good prompt; (2) a workflow, a fixed DAG of calls the developer orchestrates (classify, retrieve, draft, validate); (3) an agent, where the model itself decides which tools to call, in what order, looping until done. This matches how Anthropic's own "Building Effective Agents" guidance frames it: use the simplest thing that works.
Each rung up costs you: reliability compounds, a 5-step loop where each step is 95% reliable is about 77% reliable end-to-end; latency goes from one round-trip to many (2s to 30s and beyond); cost multiplies with every loop iteration re-sending context; debuggability collapses, because failures are now emergent paths, not fixed code; evals get harder, because the space of trajectories explodes.
The compounding is worth tabulating, because intuition fails at it (all values are the per-step rate raised to the step count):
| Steps in the loop | 95% per step | 99% per step |
|---|---|---|
| 5 | 77% | 95% |
| 10 | 60% | 90% |
| 20 | 36% | 82% |
Two readings matter. First, at 95% per step, a 20-step agent fails almost two times in three; nobody would ship a cron job with that record, and "the model is pretty reliable at each step" quietly becomes an unshippable product. Second, the table says where improvement money goes: pushing per-step reliability from 95% to 99% (tighter tools, validation between steps, narrower actions) buys far more than shortening the loop by a step or two. That is why production agents are mostly engineering around the loop, not prompting inside it.
So when is an agent right? Three conditions, roughly all required: the task has unpredictable structure (you can't enumerate the steps in advance, like open-ended research or multi-system debugging); the model has tools and feedback (it can check its own work, run the query, see the error); and the value of autonomy exceeds the cost of failure, or failures are cheap to catch (human review, sandboxes, reversible actions). Classic agent-appropriate: "investigate why this customer's invoice is wrong" across five systems. Classic agent-inappropriate: "summarize this ticket" or "extract these six fields", where fixed workflows do it cheaper, faster, and testably.
Give the consultant's move: when a customer says "we want an agent," translate it to outcomes. Run the translation on a real ask: "we want an agent to handle refund requests." Enumerate what handling one takes: classify the request, look up the order, check it against refund policy, draft the response, route edge cases to a human. Every step was just enumerated, which is the tell; this is a workflow, and each stage can be tested, evaled, and debugged on its own. The genuinely agentic residue is the 5% of tickets where the policy is ambiguous or systems disagree, and those go to the human queue anyway. Often a router plus two deterministic workflows hits 95% of the value at a tenth of the operating complexity, and you can always graduate to an agent later, with the workflow as your fallback and baseline. The customer got the outcome they wanted; the word "agent" turned out to be a requirement about results, not architecture.
What interviewers probe next
"A customer insists on an agent for a task you think needs a workflow, what do you do?" (Build the workflow as 'agent v0,' show eval numbers, let evidence argue.) "How would you know your agent is over-engineered?" (Trace logs show the same tool sequence 90% of the time: that's a workflow wearing a costume.) "Where do agents win today?" (Coding with test feedback, research and triage, long-tail support: domains with verifiable intermediate signals.)
Common mistakes
Enthusiasm without economics: pitching agents because they're impressive, not because the task demands autonomy. Failing to define terms, so "agent" stays a vibe. Never mentioning compounding error rates or eval difficulty. And missing the trap entirely: at Anthropic, Sierra, and OpenAI this question is scored on whether you reach for the simplest architecture first, not on how much agent vocabulary you own.
Key takeaways
- Default to the lowest rung of the ladder; each step up multiplies cost, latency, and eval difficulty.
- Three conditions justify an agent: unpredictable structure, tool feedback, and autonomy worth the failure cost.
- A trace that repeats the same tool sequence is a workflow in disguise; demote it.