Chain of Thought
Chain of thought is a prompting technique that encourages a large language model to generate intermediate reasoning steps before producing a final answer, improving performance on complex reasoning tasks.
Definition
Chain of thought is a prompting technique that encourages a large language model to generate intermediate reasoning steps before producing a final answer, improving performance on complex reasoning tasks.
Overview
Rather than asking a model to jump straight to an answer, chain-of-thought prompting asks it to "think step by step," breaking a problem into smaller intermediate steps the way a person might work through a math problem on paper. This has been shown to meaningfully improve accuracy on arithmetic, logic, and multi-step reasoning tasks compared to asking for a direct answer, because it gives the model more tokens and structure to work through before committing to a conclusion. Chain of thought can be elicited explicitly through prompt engineering — for example, instructing the model to "explain your reasoning" — or it can be built into a model's default behavior, as seen in reasoning-focused models that generate extended internal reasoning traces before responding. It is a form of in-context reasoning that complements, rather than replaces, techniques like few-shot learning, where example reasoning chains are shown to the model in the prompt. The technique is widely used in production systems for tasks like code generation, mathematical problem solving, and complex decision-making pipelines, including AI agents that need to plan multi-step actions. The course AI Agents & Agentic Workflows and the blog post Prompt Engineering: Get Better Results from Any LLM both cover practical techniques for using it effectively.
Key Concepts
- Elicits intermediate reasoning steps before a final answer
- Improves accuracy on arithmetic, logic, and multi-step reasoning tasks
- Can be prompted explicitly or built into a model's default reasoning behavior
- Often combined with few-shot examples showing worked reasoning chains
- Trades additional generated tokens for improved answer quality
- Widely used in agentic and multi-step planning workflows
Use Cases
Frequently Asked Questions
From the Blog
Chain-of-Thought Prompting: Make AI Reason Better
Chain-of-thought prompting asks a model to reason step by step before answering, which improves accuracy on problems that need multiple stages of logic.
Read More AI & TechnologyWhat Is Chain-of-Thought Prompting?
Chain-of-thought prompting asks an LLM to reason step by step before answering, which noticeably improves accuracy on math, logic, and multi-step problems.
Read More AI & TechnologyReAct, Chain-of-Thought and Tree of Thoughts Compared
Chain-of-thought adds reasoning tokens, ReAct interleaves reasoning with tool calls, and Tree of Thoughts explores multiple reasoning branches with backtracking. This article compares the three on cost, latency and the problem shapes where each genuinely improves accuracy, and shows how to escalate between them so that easy inputs never pay for the expensive scaffold.
Read More AI & TechnologyWhen chain-of-thought prompting helps and when it just costs tokens
Reasoning prompts help tasks with intermediate state and waste tokens elsewhere. Learn where they pay, how to hide the working, and how to measure the trade-off.
Read More