Chain-of-Thought Reasoning Explained: From Step-by-Step Prompting to Trained Reasoning Models, Faithfulness and Practical Use
How chain-of-thought reasoning evolved from a prompting trick (Wei et al., Kojima et al.) through self-consistency, Tree of Thoughts and ReAct to RL-trained reasoning models like o1 and DeepSeek-R1, where it helps, its limits, whether visible reasoning is faithful, and how to prompt reasoning models.
Chain-of-thought (CoT) reasoning, getting a language model to work through intermediate steps before answering, has gone from a prompting trick to a core design feature of modern AI. In 2022, researchers showed that worked examples or simply the phrase “Let’s think step by step” improved performance on multi-step problems. Since OpenAI’s o1 in September 2024, labs have trained models with reinforcement learning to reason at length before answering. Reasoning helps most on problems with clear structure and checkable answers, such as maths and code, but visible reasoning is not always a faithful account of how the model reached its answer, and errors can compound across steps. Verify final answers with tools rather than trusting the reasoning text.
How the techniques evolved
| Technique | Year | Idea | Source |
|---|---|---|---|
| Chain-of-thought prompting | 2022 | Include worked reasoning steps in few-shot examples | Wei et al., arXiv:2201.11903 |
| Zero-shot CoT | 2022 | Add “Let’s think step by step” without examples | Kojima et al., arXiv:2205.11916 |
| Self-consistency | 2022 | Sample several reasoning paths and take a majority vote | Wang et al., arXiv:2203.11171 |
| ReAct | 2022 | Interleave reasoning with tool actions such as search | Yao et al., arXiv:2210.03629 |
| Tree of Thoughts | 2023 | Explore and evaluate branching reasoning paths with search | Yao et al., arXiv:2305.10601 |
| RL-trained reasoning models | 2024–2025 | Train models with reinforcement learning to produce long internal reasoning | e.g. OpenAI o1; DeepSeek-R1, arXiv:2501.12948 |
Why step-by-step reasoning helps
The mechanism is not fully understood, but a common explanation is computational: generating intermediate steps gives the model more computation per problem, and each step conditions the next, so the model can break a problem into pieces and use earlier results. Self-consistency adds robustness by averaging over several reasoning paths, reducing the impact of any single faulty chain. Tree-based methods go further, generating alternative next steps, evaluating them and pursuing the most promising.
Modern systems combine reasoning with tools and verification. A model working through a maths problem can write and run code to check calculations; one answering a legal or factual question can retrieve sources to ground its steps. This scaffolding catches errors before they compound.
From prompting to trained reasoning models
Since OpenAI’s o1, major labs have released models trained with reinforcement learning to reason before answering. DeepSeek’s R1 report (see our DeepSeek-R1 guide) described training reasoning behaviour largely through RL with verifiable rewards, such as checking whether a maths answer is correct or code passes tests. These models typically expose a setting for reasoning effort, trading cost and latency for accuracy.
This changes how to prompt them. Reasoning models often perform better when given a clear goal, constraints and output format rather than step-by-step instructions, and some providers advise against forcing explicit chain-of-thought prompts with them. For standard chat models, asking for brief reasoning before the final answer still helps on multi-step problems.
Where reasoning helps most
| Domain | Why reasoning helps | Caveats |
|---|---|---|
| Mathematics | Problems decompose into steps; answers are checkable | Arithmetic slips still occur; verify with code |
| Coding and debugging | Reasoning about logic and causes of bugs improves fixes | Tests are the real check |
| Planning | Working through dependencies and constraints improves coherence | Plans can look complete while missing real-world constraints |
| Analysis of documents | Step-by-step extraction reduces missed details | Must be grounded in the source text |
Limits
Novel problems. Reasoning works best where similar patterns appear in training data. Problems requiring a genuinely new approach remain hard, and step-by-step text does not guarantee insight.
Compounding errors. Each step can introduce a mistake or fabrication that later steps build on, producing a confident but wrong conclusion. The careful structure can create an illusion of reliability.
Cost and latency. Long reasoning uses many more tokens. For simple questions, it adds expense without benefit.
Overthinking. On easy tasks, extended reasoning can sometimes lead models to second-guess correct answers.
Is the visible reasoning faithful?
Research has repeatedly found that stated reasoning does not always reflect what drives a model’s answer. Turpin et al. (2023) showed that models can give plausible explanations while being influenced by biasing features they never mention (arXiv:2305.04388). Treat chains of thought as useful for debugging and building user trust, not as a guaranteed audit trail. This matters for safety research too: monitoring a model’s reasoning is only as reliable as the reasoning is faithful.
Practical guidance
- For standard chat models, ask for brief reasoning before the final answer on multi-step problems.
- For reasoning models, specify the goal, constraints and output format; adjust the reasoning-effort setting instead of adding long instructions.
- Use self-consistency, several samples plus a vote, for high-stakes numerical answers when cost allows.
- Verify final answers with tools such as code execution, calculators or retrieval rather than trusting the reasoning text.
- Use low reasoning effort or a non-reasoning model for simple lookups and formatting tasks to save cost.
A worked example
Consider the question: “A shop sells pens at 3 for $2. How much do 12 pens cost?” Asked for an immediate answer, a model might pattern-match to a wrong figure. Prompted to reason, it writes: 12 pens is 4 groups of 3; each group costs $2; 4 × $2 = $8. The intermediate steps make the calculation explicit and easy for a reader, or a tool, to check. For harder problems, the same structure lets you see where a chain went wrong, which is why reasoning output is valuable for debugging even when it is not a perfect window into the model’s internals.
Common questions
Should I always ask models to think step by step? No. It helps on multi-step problems with standard chat models, but adds cost on simple tasks, and reasoning models already do it internally.
Do reasoning models hallucinate less? They tend to do better on structured problems, but they can still produce confident errors, sometimes embedded in long reasoning that looks convincing.
Can I see a reasoning model’s full chain of thought? It depends on the provider. Some show full reasoning, others show summaries or hide it, partly for safety and competitive reasons.
Our analysis
Chain-of-thought is one of the most important practical advances in making language models useful for complex work, and RL-trained reasoning models have made it a built-in capability rather than a prompt trick. But it is best understood as a way to give models more structured computation, not as transparent machine thinking. It is most reliable when answers can be checked and least reliable when a confident chain of plausible steps cannot be verified. The practical rule is simple: let the model reason, then check the result with something other than the model’s own words.