ReTool: Reinforcement Learning for Strategic Tool Use in LLMs

ReTool trains a language model to use a code interpreter while solving mathematical problems. The model writes reasoning, emits a program, receives the actual execution result and continues. Reinforcement learning rewards the final answer, allowing useful tool-use strategies to become more likely over training.

The process starts with supervised demonstrations so the model can already produce the required interaction format. During the later RL stage, the interpreter is part of the rollout environment. A failed program can return an error that the model uses to revise its approach; a successful program can provide a calculation that supports the next reasoning step.

The primary experiment uses Qwen2.5-32B-Instruct. It reports 67.0% on AIME 2024 after 400 optimizer steps, compared with 40.0% for a text-only RL baseline after 1,080 steps. Those are accuracy and update-count comparisons. They do not establish an equivalent reduction in wall-clock time or total compute, because tool execution changes the work performed during each rollout.

The official project provides the authors' release links. This breakdown examines the April 2025 report as evidence about learning an interaction policy, with particular attention to what the outcome reward does and does not verify.

An enterprise assistant often needs to move between prose and deterministic operations. A calculator, query engine or constrained analysis tool can produce evidence that a model cannot reliably invent. The engineering problem is deciding when to call it, how to use the result and when the result is insufficient.

ReTool makes those decisions part of training. That is relevant when a team already has repeated, measurable failures in a bounded workflow and can build trustworthy outcome checks. It cannot repair a broken integration or define success on the customer's behalf. Fix those problems before funding a training run.

The paper also helps explain a common demonstration trap: code can run without solving the right problem. The tool-calling contract checks an action's shape; the task verifier checks its meaning. Both belong beside the model, along with sandboxing appropriate to the execution environment.