Self-Consistency Improves Chain of Thought Reasoning in Language Models

Chain-of-thought prompting gets a model to write out its reasoning before answering, and it is usually decoded greedily: one path, the most likely token at each step. Xuezhi Wang, Jason Wei, Denny Zhou and colleagues at Google Research noticed that a reasoning problem can usually be solved several ways that reach the same answer, while mistakes tend to land on different wrong answers. So they replaced greedy decoding with sample and marginalize: sample many reasoning paths, parse the final answer from each, and return the most frequent one.

That is the whole method. It uses the same few-shot chain-of-thought prompt, any standard sampling scheme (they used temperature 0.5 to 0.7, top-k 40 for most models), and a parser that reads the text after "The answer is". The reasoning path is a hidden variable that gets summed out; only the answers vote.

They tested it on four models (UL2 20B, LaMDA 137B, PaLM 540B, and GPT-3's code-davinci-001 and -002) across arithmetic benchmarks (GSM8K, SVAMP, AQuA, ASDiv, MultiArith, AddSub) and commonsense ones (CommonsenseQA, StrategyQA, ARC). With 40 sampled paths, self-consistency improved on greedy chain of thought for every model on every arithmetic and commonsense task. GSM8K with PaLM 540B rose from 56.5% to 74.4%; AQuA with code-davinci-002 from 39.8% to 52.0%.

The paper also showed that voting beat ranking samples by the model's own probability, beam search, and ensembles of prompts; that it rescued tasks where chain of thought had hurt; and that the share of paths agreeing with the winner tracks accuracy closely enough to serve as a confidence score. It appeared at ICLR 2023.

Self-consistency is often the first thing to try when a customer's extraction or reasoning task is close to good enough. It needs no new data and no new model, only a parser for the final answer.

It costs one decode per path. Forty paths is forty times the output tokens and, unless you batch, more latency. The authors suggest starting with 5 or 10 paths, where most of the gain arrives.

The agreement rate is a free confidence score. If 9 of 10 paths agree, the answer is very likely right; if they split 4, 3 and 3, it is not. That is the signal for routing a case to a person, retrying with a stronger model, or asking for more information.

It needs answers that can be compared. Numbers, labels, yes or no, a field value. For free-form text you need a way to decide when two answers say the same thing before you can vote.

It is the baseline for anything fancier. Tree search, self-refinement and LLM critics all spend extra compute. A vote over the same number of samples is what they must beat.