A query that errors is a bug. A query that runs and returns a plausible wrong number is a product teaching a finance team, one figure at a time, not to trust it. The diagnosis is a classification exercise, and the fix is almost never a better prompt.
Updated Sep 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.
A query that errors is a bug. A query that runs and returns a plausible wrong number is a product teaching a finance team, one figure at a time, not to trust it. The diagnosis is a classification exercise, and the fix is almost never a better prompt.
Strong candidates refuse to touch the prompt until they have sorted the misses into classes, because the four common classes have four different fixes and only one of them lives in the model. The second thing being scored is whether they know that exact-match on SQL text is the wrong metric and that execution accuracy against a golden result set is the right one. Anyone who proposes 'add more examples to the prompt' as the first move has answered a different question.
No comments yet — be the first to share your approach.
