IN THIS BREAKDOWN · 10 SECTIONS
You can read 19% of this breakdown.
The paper in one minute
Self-consistency samples many reasoning paths for a question and takes the most common answer. On GSM8K, it lifts PaLM-540B from 56.5% with one chain-of-thought pass to 74.4% with 32. That gap suggests the model can already produce better answers than its single pass shows. This paper trains the gap into the model.
The method has three steps. Take questions from a training set and ignore their answers. For each, sample 32 chain-of-thought paths at temperature 0.7 and find the majority answer. Keep every path that reached that answer, write each in four formats (with chain-of-thought examples, with direct-answer examples, with "Let's think step by step" and no examples, with no examples at all), and fine-tune the model on the result. The authors call the trained model LMSI, Language Model Self-Improved.
Across six tasks (GSM8K, DROP, ARC-Challenge, OpenBookQA, ANLI-A2 and ANLI-A3), every prompting method improved. Self-consistency rose by 1.1 to 7.7 points, and a single greedy pass after training came close to or beat self-consistency before it. A model trained on all six tasks' questions also improved on held-out tasks, most on those similar to the training set. Training on the chain-of-thought formats mattered: direct answers alone kept less than a third of the gain. Generated questions and self-generated prompts also worked, less well.
Research notes: the problem, approach and results
GSM8K with self-consistency, 74.4% to 82.1%, using training questions but no training answers (Table 3)
one greedy chain-of-thought pass after self-training, against 74.4% for 32-path voting before it (Table 3)
of kept training paths end in a wrong answer in our simulation matched to the paper's 56.5% and 74.4% (our run)
Improving a large model's reasoning usually needs labeled data, and labeled reasoning traces are expensive. Self-consistency (sampling many reasoning paths and voting) improves accuracy at inference, but costs many samples per question every time.
Use the model's own majority vote as a pseudo-label: sample 32 chain-of-thought paths per unlabeled question, keep the paths that reach the most common answer, and fine-tune on them in four prompt formats so the model does not overfit one style.
PaLM-540B improved on all six training tasks without ground-truth labels: GSM8K 74.4% to 82.1%, DROP 78.2% to 83.0%, OpenBookQA 90.0% to 94.4%, ANLI-A3 63.4% to 67.9% under self-consistency. Gains carried to held-out tasks, most on tasks similar to training, and to smaller models trained on the same data.
Why it matters to an FDE
It cuts inference cost. If a customer is paying for self-consistency in production, training on the votes can bring one greedy pass close to the voted accuracy: 73.5% against 74.4% on GSM8K, with 1 sample instead of 32. That is the most direct business use of the paper.
It uses data customers have. Most customers have many real questions and few labeled answers. Majority-vote pseudo-labels let you adapt a model to their question distribution with no annotation budget, as long as the model is already right more often than not on many of them.
It repeats the model's mistakes. When the majority is wrong, training teaches the wrong answer more firmly. Keep a labeled sample to measure pseudo-label accuracy, filter on vote confidence, and check that the fine-tuned model does not get more confident on its systematic errors.
It transfers to smaller models. The same self-generated data lifted PaLM-62B above the untrained 540B model on GSM8K, which is the path from an expensive model to a cheaper one.
Understand the mechanism. Try it on a real problem.
Unlock this breakdown and 290 more papers.
Premium opens every paper in full, plus every answer, concept, course and guide on the site. One payment for six months of access.
6 months · One payment · No auto-renewal
- ◇1 architecture diagram
- </>1 runnable code experiment
- ▦2 results tables
- ?5 interview questions with answers
- 03Key takeaways
The pipeline, why mixed formats, the gains and their limits, distillation and two misreadings
Free account - 04How it works
Sampling and voting, the four formats, training settings, generated questions and prompts
Premium - 05The numbers that matter
Table 3 before and after, held-out tasks, formats, generated data, distillation and voting
Premium - 06Try it yourself
Read the tables for what changed, then simulate what majority-vote labels contain
Premium - 07Where it breaks
Benchmark questions as inputs, temperature tuned on test data, one model, wrong majorities, slips
Premium - 08Using it on a deployment
When to self-train, how to filter pseudo-labels, and how to check you did not amplify errors
Premium - 09Interview angles
Five questions this paper prepares you for, with answer hints
Premium - 10Read next
Where to go from here on this site
Premium