Explain why finance's SQL is wrong
A text-to-SQL pilot is live for the finance team. Asked for revenue by region for Q2, it produces a query that runs without error and returns APAC 6,000 and EMEA 2,400. Finance says both numbers are wrong. They cannot say why, only that the totals do not match their close.
The real answer is APAC 3,500 and EMEA 2,000, and getting there means finding three separate defects in one short query. One inflates the total, two deflate it, and because they partly cancel, the finished number looks plausible enough that nobody caught it for a quarter.
You get the schema, the seed data, the query the model wrote, and a grader that runs on Python's standard library with nothing to install. The SQL is the smaller half of the exercise. The half that decides the round is saying what the output actually proves, which is less than people assume: two regional totals contain no individual order, so none of the three is visible from the numbers alone. Then what each defect takes to find, and why a precise prompt can produce the right query once and still not prevent the next five hundred.
What is being graded
- What the result set proves on its own, and what it cannot: none of the three defects is visible in two regional totals
- Which check exposes each defect, since a row count across the join finds one and only knowing the data finds the others
- Why a precise prompt can produce the right query once and still not prevent the next five hundred
- A note to finance that carries a number and a decision, not a schema tour
The brief above is the whole problem, and you are welcome to build the fixtures yourself. The pack saves you that hour: schema, seed data, the broken version, a starter file, a grader, and a reference solution that explains what was being scored rather than just showing the answer.
See premium access →Background reading: text to sql runs but wrong triage