A crashed agent is easy. The hard one returns a confident answer after a tool quietly returned nothing and the model made up the rest. The strong answer instruments the steps that fail without throwing, ties traces to an outcome signal, and refuses to call a 200 a success.
Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.
A crashed agent is easy. The hard one returns a confident answer after a tool quietly returned nothing and the model made up the rest. The strong answer instruments the steps that fail without throwing, ties traces to an outcome signal, and refuses to call a 200 a success.
This separates people who have debugged agents in production from people who have read about tracing. The naive answer logs the prompt and the final response and calls it observability; it cannot tell you that step 3's retrieval returned zero rows and the model confabulated around the gap. The discriminating instinct is that the dangerous failures are the ones that do not raise an exception: empty tool results, truncated context, a guardrail that rewrote the output, plan drift where the agent picks the wrong branch. The follow-up that scores: 'a user reports a wrong answer, walk the trace.' A strong candidate names span attributes (tool args, raw output, retry count, token usage, finish_reason) and the outcome signal (eval score, user thumbs, downstream correction) that lets them find these runs at scale instead of one ticket at a time. Saying 'we use OpenTelemetry' without naming what to put on the spans is a tell.
No comments yet — be the first to share your approach.
