DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

DeepSeekMath is remembered for GRPO, the reinforcement-learning method later used to train DeepSeek-R1, but the paper is a full recipe for a math-specialist model with three stages.

First, data. The team trained a fastText classifier on OpenWebMath pages as positives and random web pages as negatives, ran it over 40B deduplicated Common Crawl pages, and kept the highest-scoring ones. They then found domains where many pages had been selected, had people mark the math sections of those sites, added those pages to the positive set and retrained. After four rounds, the DeepSeekMath Corpus held 35.5M pages and 120B tokens. In a controlled test with a 1.3B model, it beat existing open math corpora on every benchmark. DeepSeek-Coder-Base-v1.5 7B was then trained on 500B tokens, 56% from this corpus. The result, DeepSeekMath-Base 7B, scored 64.2% on GSM8K and 36.2% on MATH, above the 540B Minerva.

Second, instruction tuning on 776K math problems with chain-of-thought, program-of-thought and tool-using solutions, which brought MATH to 46.8%. Third, GRPO. PPO, the usual RL algorithm for language models, trains a value model as large as the policy to judge how good each partial answer is. GRPO drops it: for each question it samples 64 answers, scores them with a reward model, and treats each answer's reward relative to the group's average and spread as its advantage. Trained on 144K GSM8K and MATH questions only, the model reached 88.2% on GSM8K and 51.7% on MATH, and it improved on Chinese math benchmarks it never saw during RL. The authors' analysis found that RL made correct answers more likely to be sampled but did not let the model solve problems it could not solve before.

GRPO is now the default starting point for reinforcement learning on language models with checkable answers, so customers asking about fine-tuning with rewards are usually asking about this paper's method.

GRPO halves the models you have to train. Dropping the value model removes a policy-sized network and its optimizer state. By a standard accounting, training memory for a 7B model falls from about 252GB to 140GB before activations. That puts RL fine-tuning within reach of teams with a few GPUs and a reward they can compute.

RL sharpens, it does not teach. The model's majority-vote accuracy rose while its Pass@K (whether any of K samples is right) did not. RL made the model more reliably produce answers it could already reach. If a customer's base model never solves a class of problems, RL on outcome rewards is unlikely to fix it; better data or a stronger base model will.

Domain data can be mined, iteratively. The corpus pipeline (seed classifier, score the web, find the domains that light up, have people label the right URL paths, retrain) works for any domain with a seed set: legal filings, clinical guidelines, internal wikis. The 1.3B comparison shows quality and quantity together beat smaller curated sets.