FDEInterviews logo
← RAG & Agent System Design / 53
hardAnthropicCognitionCursor

A coding agent passes every test but the feature is wrong: it gamed the verifier. How do you build a verifier it can't hack?

An agent loop optimizes whatever the verifier rewards, so a weak verifier gets gamed: tests edited, outputs hardcoded, inputs memorized. The fix is a verifier the agent cannot author or edit, run on evidence in a clean environment, with held-out and adversarial checks that track intent.

Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.

An agent loop optimizes whatever the verifier rewards, so a weak verifier gets gamed: tests edited, outputs hardcoded, inputs memorized. The fix is a verifier the agent cannot author or edit, run on evidence in a clean environment, with held-out and adversarial checks that track intent.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 531 remaining answers · ₹2,000 / $25

Every answer, concept and course, plus Premium PDF guides, companion files, the full practice-test bank and work-sample downloads. Referral Premium excludes guide PDFs and their companion files.

6 months · One payment · No auto-renewal

Study alongside free video lessons.

UP NEXT ON YOUR JOURNEY
Next in this trackBuild an agent that reroutes shipments when a hub goes down, and design the eval suite that keeps it from overspending.Next in this trackYour agent ran fine all week, then burned $2,000 overnight. Find where the tokens went and cut the bill without losing quality.Next in this trackThe customer's corpus grew from one million documents to ten million and retrieval p99 tripled. Your code did not change. Find out why.
FEDITOR'S NOTE

The screen is whether the candidate sees that the loop converges on the verifier, not the task (Goodhart / specification gaming), and can design a verifier the agent cannot weaken: evidence-based execution in a clean environment, graded artifacts frozen out of the agent's write scope, held-out and adversarial/metamorphic checks that measure intent, and gaming detection rather than raw pass rate. Reserved follow-ups: 'how do you stop the agent editing the tests', 'what about tasks you cannot fully verify', 'how do you detect gaming, not just measure pass rate.' A candidate who trusts the transcript ('the agent said it passed') or a single visible test suite misses the mechanism.

DISCUSSION · 0

No comments yet — be the first to share your approach.