Plain-English Summary
When you ask a large language model a math question, it often gets the wrong answer by trying to jump straight to the solution. But if you show it examples where the solution includes step-by-step reasoning, the model learns to “think out loud” and its accuracy improves dramatically. This is chain-of-thought prompting.
The technique is surprisingly simple. Instead of showing the model “Q: What is 23 x 17? A: 391,” you show it examples like “Q: What is 23 x 17? A: Let me work through this. 23 x 17 = 23 x 10 + 23 x 7 = 230 + 161 = 391.” When the model sees this pattern, it generates its own step-by-step reasoning for new problems.
Think of it as the difference between asking someone for an answer versus asking them to show their work. Showing work forces careful, sequential thinking rather than gut reactions.
Key Innovation
The paper demonstrated that including intermediate reasoning steps in few-shot examples causes models to generate their own reasoning chains, substantially improving accuracy on tasks requiring multi-step logic. On the GSM8K math benchmark, chain-of-thought prompting improved accuracy from 17.9% to 58.1% for PaLM 540B — a massive gain from a simple prompting change.
Crucially, this capability is emergent with scale. Chain-of-thought prompting helps large models significantly but provides little or no benefit to smaller models. This suggests that the ability to perform step-by-step reasoning is latent in large models and just needs to be elicited through the right prompting strategy.
Impact on the Field
Chain-of-thought prompting transformed how people interact with language models. The idea that you can dramatically improve model capabilities simply by asking it to reason step-by-step became foundational knowledge for prompt engineering. It also shifted the field’s understanding of what capabilities are “hidden” inside large models, waiting to be unlocked.
The technique directly influenced model training. Modern models like GPT-4 and Claude are trained to produce chain-of-thought reasoning by default on complex tasks. DeepSeek-R1 and similar models use reinforcement learning to optimize reasoning chains, extending the core insight into the training process itself.
Models That Built on This
GPT-4 and Claude incorporate chain-of-thought reasoning as a core capability. Google’s PaLM 2 and Gemini were evaluated extensively on chain-of-thought tasks. DeepSeek-R1 used RL to train models to produce better reasoning chains. The “thinking” or “extended thinking” features in modern AI assistants are direct descendants of this work. Self-consistency (sampling multiple reasoning chains and voting) and tree-of-thought (exploring branching reasoning paths) are extensions that build on the same foundation.