Before you touch a model, check whether the two numbers measure the same thing. Field accuracy of 96% across twelve fields is 61% of documents fully correct, which means half of the drop may be arithmetic. The other half is the documents the pilot never saw.
Updated Sep 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.
Before you touch a model, check whether the two numbers measure the same thing. Field accuracy of 96% across twelve fields is 61% of documents fully correct, which means half of the drop may be arithmetic. The other half is the documents the pilot never saw.
The first thing being scored is whether the candidate asks what 95% and 60% each count before explaining either. Most candidates jump to 'production data is messier', which is true and incomplete. The second thing is the order of fixes: parse before model, and a per-field confidence gate before any of it, because the customer's real question is not 'how accurate is it' but 'which fields can I stop checking'.
No comments yet — be the first to share your approach.
