GO TO A SECTION

↑ ↓ to navigate · Enter to openExplore all sections →
FDEInterviews logo
arXiv

Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipeline

A solve, self-review, verify and fix loop took Gemini 2.5 Pro, Grok-4 and GPT-5 to 5 of 6 IMO 2025 problems, against best-of-32 baselines of 21% to 38%. Attempt counts and grading go unreported.

Breakdown by FDE Interviews · 12 min read · Updated

About the original · 2025
AUTHORS
Huang, Yang
FROM
UCLA
PUBLISHED
Jul 2025 · arXiv
CITED BY
77Semantic Scholar · Nov 2026
IN THIS BREAKDOWN · 10 SECTIONS

You can read 25% of this breakdown.

01

The paper in one minute

International Mathematical Olympiad problems require complete, rigorous proofs of novel problems, under a selection process designed to avoid anything too similar to known problems. Independent grading by MathArena found that leading models in mid-2025 scored 31.6% (Gemini 2.5 Pro), 21.4% (Grok-4) and 38.1% (GPT-5) of the points on IMO 2025, even when each model chose its best answer out of 32 attempts.

This paper argues that the models have more ability than single attempts show, and that a structured loop can draw it out. The pipeline uses one model in every role. Step 1 generates a solution with a prompt that stresses rigor and allows the model to report partial results instead of guessing. Step 2 asks the model to review and improve its own work, which the authors describe as giving it a second budget of reasoning tokens. Step 3 runs the same model as a verifier with a strict grading prompt: it must check every step and list each issue as a critical error or a justification gap. Step 4 optionally reviews the bug report to remove false alarms. Step 5 asks the model to correct the solution using the report, and the loop returns to verification. A solution is accepted when it passes verification five times in a row and rejected after repeated major issues.

With any of the three models the pipeline produced proofs for Problems 1 to 5 and failed Problem 6, a combinatorics problem on a 2025 by 2025 grid; the authors identify the flaw in the failed attempt. Early versions used general hints for Problems 1 and 2 (induction, analytic geometry); later versions solved them without hints at higher cost. The full proofs are printed in the appendix as evidence.

MathArena later ran the authors' public code with Gemini 2.5 Pro on the IMC 2025 university competition, held after the code was released, and scored it 94.5%, compared with 57.7% for the model alone.

THE IDEA, VISUALLY
THE LOOP (SECTION 2.1) 1. Generate solution 2. Self-improve second reasoning budget 3. Verify critical errors, gaps 4. Review bug report optional 5. Correct solution loop Accept 5 clean in a row Reject 10 failing rounds One model plays every role. No tools, no web. Runs per problem and tokens spent are not reported. If the verifier's misses are shared with the writer, five passes do not remove them. arXiv 2507.15855 v4, UCLA REPORTED RESULTS (UNITS DIFFER) IMO 2025, best-of-32 baseline, % of points (MathArena) Gemini 2.5 Pro 31.6% Grok-4 21.4% GPT-5 38.1% IMO 2025 with the pipeline, problems solved Gemini 2.5 Pro P1 P2 P3 P4 P5 P6 Grok-4 P1 P2 P3 P4 P5 P6 GPT-5 P1 P2 P3 P4 P5 P6 5 of 6 = 83.3%; the paper writes 'approximately 85.7%' IMC 2025, Gemini 2.5 Pro, % (MathArena, independent) model alone 57.7% in the pipeline 94.5% rank 92 -> rank 3 of 434 human participants
Left: the pipeline from Section 2.1. A solution is generated, self-improved, then verified and corrected in a loop; it is accepted after five consecutive clean verifications and rejected after ten rounds with major issues. Right: results as reported, with units kept apart. MathArena's best-of-32 baselines are percentages of points; the pipeline's result is problems solved; the IMC 2025 figures come from MathArena's independent run.
Research notes: the problem, approach and results
5 of 6

IMO 2025 problems solved with each of three base models; the paper also states this as 85.7%, though 5/6 is 83.3% (our check)

21-38%

MathArena's best-of-32 baselines for the same models, as a share of points (Section 1)

57.7% -> 94.5%

IMC 2025 score with Gemini 2.5 Pro alone vs inside the pipeline, run independently by MathArena (Section 4.2)

THE PROBLEM

Frontier models do well on answer-only math benchmarks but struggle to write complete, rigorous proofs for olympiad problems; independent graders found leading models below bronze on IMO 2025. Is the limit the models, or how they are used?

THE IDEA

Wrap one model in prompts for six steps: generate a rigorous solution, self-improve it, have the same model act as a strict verifier that writes a bug report of critical errors and justification gaps, optionally review that report, correct the solution, and repeat. Accept a proof only after five consecutive clean verifications.

THE RESULT

With Gemini 2.5 Pro, Grok-4 or GPT-5 as the model, the pipeline produced proofs for 5 of the 6 IMO 2025 problems and failed Problem 6 with all three. MathArena's independent run on the IMC 2025 university competition scored the pipeline 94.5% with Gemini 2.5 Pro, against 57.7% for the model alone.

02

Why it matters to an FDE

The pattern transfers. Generate, critique against a strict rubric, repair and accept only after repeated clean checks is a design you can apply to contracts, financial models, code changes and reports. The paper's prompts are a good template: a verifier that must classify every issue, quote where it occurs and is forbidden from fixing what it finds.

Acceptance rules matter more than prompts. The pipeline accepts only after five consecutive clean verifications. That rule is what turns a fallible checker into a usable gate, as long as its misses are independent. When the same model writes and checks, some of its blind spots are shared, and repeating the check does not remove them. A different model or a deterministic check (tests, a solver, a calculation) as one of the verifiers is the stronger design.

Report cost and attempts. The paper runs the pipeline "some number of times" and does not say how many, or how many tokens were spent per problem. A customer comparing this approach with a single call needs both: success rate and cost per accepted output.

Separate the evaluator from the system. The strongest evidence here is the independent IMC run, not the authors' own IMO claims. For any loop that grades its own work, insist on an outside check on a sample.

Verifier bug reports are useful output. Even when the loop fails, the list of critical errors and gaps tells a reviewer where to look. In professional workflows that can be more valuable than the final draft.

THE FULL BREAKDOWN

Understand the mechanism. Try it on a real problem.

Unlock this breakdown and 290 more papers.

Premium opens every paper in full, plus every answer, concept, course and guide on the site. One payment for six months of access.

6 months · One payment · No auto-renewal

Get Premium · ₹2,000 / $25
  • ◇1 architecture diagram
  • </>1 runnable code experiment
  • ▦2 results tables
  • ?5 interview questions with answers
  1. 03Key takeaways

    The six steps, the verifier's two issue types, acceptance rules, results, hints, and two misreadings

    Free account
  2. 04How it works

    The solver and verifier prompts, the loop, the acceptance rule, model settings and grading

    Premium
  3. 05The numbers that matter

    Problems solved, the 85.7% figure, baselines, IMC results, settings and what is not reported

    Premium
  4. 06Try it yourself

    Check the headline arithmetic, model the five-pass acceptance rule, and price unreported attempts

    Premium
  5. 07Where it breaks

    Six problems, unreported attempts and cost, self-grading, an unexplained percentage, hints

    Premium
  6. 08Using it on a deployment

    A generate-verify-fix loop for high-stakes documents: acceptance rules, diverse checkers, cost

    Premium
  7. 09Interview angles

    Five questions this paper prepares you for, with answer hints

    Premium
Back to the paper library →