Chain-of-Thought Prompting Paper Explained
Introduction
GPT-3Click to read tutorial: GPT-3 Paper Explained: Language Models Are Few-Shot Learners showed that large language models can learn tasks from a few examples in their prompt. But on problems which need several steps of reasoning - maths word problems, for example - even the largest models did poorly, and making them bigger helped surprisingly little. The 2022 paper Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, from Google, found a remarkably simple fix: write the examples in the prompt so that they show the intermediate reasoning steps, not just the final answers. The model then writes out its own reasoning before answering, and gets far more problems right. This post explains the chain-of-thought prompting paper with diagrams. We will compare standard and chain-of-thought prompts, go through the results on arithmetic, commonsense and symbolic reasoning, see why the technique only works in large models, and look at the follow-up work it inspired - including self-consistency, which we will implement in Java.The paper: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Authors: Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou (Google Research, Brain Team)
Published: January 2022 - NeurIPS 2022
Authors: Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou (Google Research, Brain Team)
Published: January 2022 - NeurIPS 2022
Standard prompting vs chain-of-thought prompting
A chain of thought is a series of short, natural-language reasoning steps which lead to the final answer - the way a person would work through a problem on paper. In chain-of-thought prompting, each few-shot example in the prompt is a triple: the question, a chain of thought, and the answer. Nothing else changes - the model is not trained or fine-tuned.The same few-shot prompt, with and without reasoning steps in the example.
- More computation for harder problems: a language model does a fixed amount of computation per tokenClick to read tutorial: LLM Tokens, Context Windows, Embeddings and Hallucinations it generates. Producing the reasoning as extra tokens lets the model spend more computation on problems which need more steps.
- Decomposition: a multi-step problem is broken into single steps, each of which is easy for the model.
- Interpretability: the chain shows how the model reached its answer, which helps to find where it went wrong - although the paper notes that a chain of thought is not guaranteed to reflect what the model actually computed.
- Generality: it works for any task a person could solve by reasoning in language, with no special training data or architecture.
Results
The authors tested chain-of-thought prompting on several model families - GPT-3, LaMDA, PaLM, UL2 and Codex - using 8 hand-written examples for the maths benchmarks. The headline result is on GSM8K, a benchmark of grade-school maths word problems -Accuracy on GSM8K, as reported in the paper. Chain-of-thought prompting alone beat a model fine-tuned for the task.
- Arithmetic reasoning: on GSM8K, PaLM 540B went from 17.9% to 56.9% - a new state of the art, beating a GPT-3 model which had been fine-tuned on the task and paired with a separate verifier model. Large gains were also seen on other maths benchmarks such as SVAMP and MAWPS.
- Commonsense reasoning: it helped on StrategyQA (questions needing implicit multi-step reasoning), date understanding and sports understanding, where PaLM 540B scored 95.4%, above the 84% of an unaided sports enthusiast.
- Symbolic reasoning: on toy tasks such as concatenating the last letters of words, or tracking whether a coin is still heads up after people flip it, chain-of-thought prompting let models handle longer inputs than any shown in the examples.
An emergent ability of model scale
The most striking finding is that chain-of-thought prompting only helps large models. For models below roughly 100 billion parameters, it did not help and could even hurt: small models wrote fluent chains of thought which were illogical, and reached wrong answers. Above that size, accuracy jumped. The paper calls this an emergent ability - one which is absent in smaller models and appears at scale, and which the smooth curves of the scaling lawsClick to read tutorial: Scaling Laws for Language Models (and Chinchilla) Explained would not have predicted.The authors also checked that it is really the reasoning which helps. Prompts with only an equation instead of the reasoning in words, with dots instead of reasoning (to give the model extra tokens without content), or with the reasoning placed after the answer all did much worse than a real chain of thought before the answer.
Follow-up work
- Zero-shot chain of thought: a few months later, Kojima et al. found that simply adding "Let's think step by step" after the question, with no examples at all, also triggers step-by-step reasoning and gives large gains. How to use it alongside other techniques is shown in prompt engineering techniquesClick to read tutorial: Prompt Engineering Techniques Explained with Examples.
- Self-consistency: Wang et al. (2022) sampled many different chains of thought for the same question and took a majority vote on their final answers. Correct reasoning paths tend to agree on the same answer, while mistakes scatter, so voting improved GSM8K accuracy by a further 17.9 percentage points.
- Reasoning and acting: ReActClick to read tutorial: ReAct Paper Explained: Reasoning and Acting in LLMs interleaved chains of thought with actions such as searching the web - the basis of AI agents.
- Reasoning models: models such as OpenAI's o1 and DeepSeek-R1Click to read tutorial: DeepSeek-R1 Paper Explained: Reasoning via RL are trained to produce long chains of thought before answering, rather than being prompted to.
Self-consistency in Java
Self-consistency is easy to implement around any model: ask the same question several times, let the model reason differently each time, and keep the answer most chains agree on -Self-consistency on the library question: five sampled chains of thought, one vote.
Majority vote over sampled chains of thought
//SelfConsistency.java
import java.util.*;
import java.util.regex.*;
import java.util.stream.*;
public class SelfConsistency {
static final Pattern ANSWER = Pattern.compile("The answer is (-?\\d+)");
static Optional<Integer> finalAnswer(String chain) {
Matcher m = ANSWER.matcher(chain);
return m.find() ? Optional.of(Integer.parseInt(m.group(1))) : Optional.empty();
}
public static void main(String[] args) {
// Five chains of thought sampled for: "A library has 120 books. It lends 45 and receives 30. How many now?"
List<String> chains = List.of(
"120 - 45 = 75 books left. 75 + 30 = 105. The answer is 105.",
"It lends 45, so 120 - 45 = 75. Then it receives 30: 75 + 30 = 105. The answer is 105.",
"120 + 45 = 165, and 165 + 30 = 195. The answer is 195.",
"Change is -45 + 30 = -15. 120 - 15 = 105. The answer is 105.",
"120 - 45 = 85. 85 + 30 = 115. The answer is 115.");
Map<Integer, Long> votes = chains.stream()
.map(SelfConsistency::finalAnswer)
.flatMap(Optional::stream)
.collect(Collectors.groupingBy(a -> a, TreeMap::new, Collectors.counting()));
System.out.println("Votes per answer: " + votes);
int majority = Collections.max(votes.entrySet(), Map.Entry.comparingByValue()).getKey();
System.out.println("Had we sampled only chain 3: " + finalAnswer(chains.get(2)).orElseThrow());
System.out.println("Self-consistency answer: " + majority);
}
}
Votes per answer: {105=3, 115=1, 195=1}
Had we sampled only chain 3: 195
Self-consistency answer: 105
Explanation of the code
- Each chain ends with "The answer is N" - the format taught by the examples in the prompt - which makes the final answer easy to extract with a regular expression.
- The three correct chains reach 105 by different routes, while the two wrong ones make different mistakes (adding instead of subtracting, and a subtraction error) and so disagree with each other. The vote picks 105.
- With a real model, the chains would come from calling the model several times with a sampling temperatureClick to read tutorial: LLM Temperature, Top-k and Top-p Sampling Explained above 0. The cost is one model call per chain, which is the price of the extra accuracy.
Summary
In this post we went through the chain-of-thought prompting paper. We saw that putting reasoning steps into the few-shot examples of a prompt makes a large language model reason step by step before answering, and why this helps - more computation, decomposition and interpretability. We went through the results on arithmetic, commonsense and symbolic reasoning, including PaLM 540B's jump from 17.9% to 56.9% on GSM8K, and the finding that the technique only works in models of around 100 billion parameters or more. Lastly, we looked at the follow-up work, from "Let's think step by step" to reasoning models, and implemented self-consistency's majority vote in Java. In the next post, we will see how GPT-3 was trained to follow instructions, in the InstructGPT paperClick to read tutorial: InstructGPT Paper Explained: Training LLMs with RLHF.Continue Reading: AI and LLM Papers Explained
InstructGPT Paper Explained: Training LLMs with RLHFClick to read tutorial: InstructGPT Paper Explained: Training LLMs with RLHF LoRA Paper Explained: Low-Rank Adaptation of LLMsClick to read tutorial: LoRA Paper Explained: Low-Rank Adaptation of LLMs Attention Is All You Need: The Transformer Paper ExplainedClick to read tutorial: Attention Is All You Need: The Transformer Paper Explained BERT Paper Explained: Bidirectional TransformersClick to read tutorial: BERT Paper Explained: Bidirectional Transformers Scaling Laws for Language Models (and Chinchilla) ExplainedClick to read tutorial: Scaling Laws for Language Models (and Chinchilla) Explained GPT-3 Paper Explained: Language Models Are Few-Shot LearnersClick to read tutorial: GPT-3 Paper Explained: Language Models Are Few-Shot Learners RAG Paper Explained: Retrieval-Augmented GenerationClick to read tutorial: RAG Paper Explained: Retrieval-Augmented Generation ReAct Paper Explained: Reasoning and Acting in LLMsClick to read tutorial: ReAct Paper Explained: Reasoning and Acting in LLMs DeepSeek-R1 Paper Explained: Reasoning via RLClick to read tutorial: DeepSeek-R1 Paper Explained: Reasoning via RL
InstructGPT Paper Explained: Training LLMs with RLHFClick to read tutorial: InstructGPT Paper Explained: Training LLMs with RLHF LoRA Paper Explained: Low-Rank Adaptation of LLMsClick to read tutorial: LoRA Paper Explained: Low-Rank Adaptation of LLMs Attention Is All You Need: The Transformer Paper ExplainedClick to read tutorial: Attention Is All You Need: The Transformer Paper Explained BERT Paper Explained: Bidirectional TransformersClick to read tutorial: BERT Paper Explained: Bidirectional Transformers Scaling Laws for Language Models (and Chinchilla) ExplainedClick to read tutorial: Scaling Laws for Language Models (and Chinchilla) Explained GPT-3 Paper Explained: Language Models Are Few-Shot LearnersClick to read tutorial: GPT-3 Paper Explained: Language Models Are Few-Shot Learners RAG Paper Explained: Retrieval-Augmented GenerationClick to read tutorial: RAG Paper Explained: Retrieval-Augmented Generation ReAct Paper Explained: Reasoning and Acting in LLMsClick to read tutorial: ReAct Paper Explained: Reasoning and Acting in LLMs DeepSeek-R1 Paper Explained: Reasoning via RLClick to read tutorial: DeepSeek-R1 Paper Explained: Reasoning via RL