Chain-of-Thought Prompting
Chain-of-thought prompting is a technique that elicits step-by-step intermediate reasoning from a language model — either by instructing it to 'think step by step' or by providing worked examples that reason before answering — which has…
Definition
Chain-of-thought prompting is a technique that elicits step-by-step intermediate reasoning from a language model — either by instructing it to 'think step by step' or by providing worked examples that reason before answering — which has been shown to substantially improve performance on tasks requiring arithmetic, logic, or multi-step reasoning.
Overview
When a language model is asked to jump directly to a final answer for a complex problem, it effectively must produce the correct answer in a single, unbroken burst of computation constrained by its fixed per-token processing, which is especially error-prone for multi-step arithmetic, logical deduction, or planning problems. Chain-of-thought (CoT) prompting, introduced in a widely cited 2022 Google Research paper by Jason Wei and colleagues, showed that prompting a sufficiently large language model to generate intermediate reasoning steps before its final answer — either via explicit instruction or by including a few worked examples in the prompt that demonstrate step-by-step reasoning — significantly improves accuracy on tasks like grade-school math word problems, commonsense reasoning, and symbolic manipulation, compared to prompting for a direct answer. A simplified and highly influential variant, 'zero-shot chain-of-thought,' introduced shortly after by Takeshi Kojima and colleagues, showed that simply appending a phrase like 'Let's think step by step' to a prompt — with no worked examples at all — was often sufficient to trigger substantial reasoning improvements, making the technique trivially easy to apply. The mechanism is generally understood as giving the model more intermediate computation and 'working memory' in the form of generated tokens: because each generated token can attend to all previous tokens, writing out intermediate steps effectively lets the model use additional forward passes to build toward an answer, rather than compressing an entire multi-step derivation into the computation of a single token. Chain-of-thought prompting was an important precursor to today's dedicated 'reasoning models' (such as OpenAI's o1/o3 series and DeepSeek-R1), which extend the same core idea — reasoning before answering — by training the model via reinforcement learning specifically to produce long, effective internal reasoning traces, rather than relying purely on prompting a general-purpose model. CoT prompting is also foundational to related techniques like self-consistency (sampling multiple chain-of-thought reasoning paths and taking a majority vote on the final answer) and Tree of Thoughts (exploring and backtracking across multiple reasoning branches rather than following a single linear chain).
Key Concepts
- Elicits explicit step-by-step intermediate reasoning before a final answer
- Introduced in a 2022 Google Research paper by Jason Wei et al.
- Zero-shot variant works via a simple instruction like 'Let's think step by step'
- Substantially improves accuracy on arithmetic, logic, and multi-step reasoning tasks
- Benefit generally scales with model size — smaller models show less improvement from CoT
- Provides the model more effective 'working memory' via generated intermediate tokens
- Precursor to dedicated reasoning models like OpenAI o1/o3 and DeepSeek-R1
- Foundation for related techniques like self-consistency and Tree of Thoughts
Use Cases
Frequently Asked Questions
From the Blog
Chain-of-Thought Prompting: Make AI Reason Better
Chain-of-thought prompting asks a model to reason step by step before answering, which improves accuracy on problems that need multiple stages of logic.
Read More AI & TechnologyWhat Is Chain-of-Thought Prompting?
Chain-of-thought prompting asks an LLM to reason step by step before answering, which noticeably improves accuracy on math, logic, and multi-step problems.
Read More AI & Technology7 Tasks Where Fine-Tuning Beats Prompting
Fine-tuning wins where the behaviour you need is hard to describe but easy to demonstrate, where a long prompt is paid on every call, or where a smaller model must hit a latency budget. This names seven task shapes that qualify, the signals that identify them, and the cases where prompting remains the better answer.
Read More AI & TechnologyPrompting Reasoning Models vs Standard Chat Models
Reasoning models and standard chat models want different prompts. Scaffolds that reliably improve a chat model — think step by step, numbered plans, worked exemplars — often add nothing to a model that reasons internally and can actively degrade it. Learn what to keep, what to strip, and how to route between the two.
Read More