SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution

DeepSeek-R1 showed that reinforcement learning (RL) with simple rule-based rewards can improve reasoning on math and competition coding, where answers are easy to check. SWE-RL asks whether the same works for everyday software engineering, where checking a fix normally means running a project's tests in a working environment.

The authors avoid execution during training. From GitHub they collect pull requests that link an issue, describe a bug fix and change code files, and keep 273,000 as seeds. Each seed gives the model an issue, the complete contents of the changed files plus some related unchanged ones, and keeps the patch that was actually merged as the oracle. The model writes its reasoning and then search-and-replace edits. The reward is -1 if the output is malformed, and otherwise the sequence similarity between the predicted patch and the oracle patch, computed by Python's difflib.SequenceMatcher, a number from 0 to 1. Training uses GRPO, which compares the rewards of 16 attempts at the same problem.

The result, Llama3-SWE-RL-70B, solves 41.0% of SWE-bench Verified inside Agentless Mini, a simplified pipeline that finds the relevant files, samples many repairs, writes reproduction tests and reranks. Only the repair step was trained; the model also handles file localization and test writing it was never trained for. A supervised baseline built from the same seeds scored 36.2%.

The authors also report gains outside software issues (code reasoning, math), and argue RL generalizes where supervised fine-tuning narrows.

Many customers ask whether their own engineering history (years of tickets, pull requests and merged fixes) can make a coding model better at their codebase. SWE-RL is the clearest worked recipe for turning that history into training data without building an executable environment for every repository. It also shows the price of that shortcut.

The data is cheap; the reward is a proxy. Similarity to the merged patch rewards writing what the original developer wrote. A correct fix in a different style scores lower than a wrong fix that copies the oracle's shape. The authors list this limitation themselves.

The headline uses heavy sampling. 41.0% is pass@1 only in the sense that one patch is submitted; it was chosen from 500 samples with 30 generated tests per issue. At 20 samples the same model scored 33.6%. Budget for that inference if you copy the setup.

Training is not small. One run used 512 H100s for about 32 hours, 16,384 GPU-hours, on a 70B model.

Formatting is half the battle. The base model produced correctly formatted edits only 12.2% of the time. Some of the RL gain, and much of the out-of-domain math gain, is the model learning to follow output formats.

If a customer wants this, the practical version is smaller: a modest model, their own pull requests as seeds, an execution check wherever tests exist, and similarity only as a fallback.