Chain-of-Thought Reasoning Explained: From Step-by-Step Prompting to Trained Reasoning Models, Faithfulness and Practical Use

How chain-of-thought reasoning evolved from a prompting trick (Wei et al., Kojima et al.) through self-consistency, Tree of Thoughts and ReAct to RL-trained reasoning models like o1 and DeepSeek-R1, where it helps, its limits, whether visible reasoning is faithful, and how to prompt reasoning models.

PUBLISHED // UPDATED
DURATION // 06 MIN READ
Chain-of-Thought Reasoning Explained: From Step-by-Step Prompting to Trained Reasoning Models, Faithfulness and Practical Use
FIG.01 // VISUAL DOCUMENTATION Chain-of-Thought Reasoning Explained: From Step-by-Step Prompting to Trained Reasoning Models, Faithfulness and Practical Use

Chain-of-thought (CoT) reasoning, getting a language model to work through intermediate steps before answering, has gone from a prompting trick to a core design feature of modern AI. In 2022, researchers showed that worked examples or simply the phrase “Let’s think step by step” improved performance on multi-step problems. Since OpenAI’s o1 in September 2024, labs have trained models with reinforcement learning to reason at length before answering. Reasoning helps most on problems with clear structure and checkable answers, such as maths and code, but visible reasoning is not always a faithful account of how the model reached its answer, and errors can compound across steps. Verify final answers with tools rather than trusting the reasoning text.

How the techniques evolved

TechniqueYearIdeaSource
Chain-of-thought prompting2022Include worked reasoning steps in few-shot examplesWei et al., arXiv:2201.11903
Zero-shot CoT2022Add “Let’s think step by step” without examplesKojima et al., arXiv:2205.11916
Self-consistency2022Sample several reasoning paths and take a majority voteWang et al., arXiv:2203.11171
ReAct2022Interleave reasoning with tool actions such as searchYao et al., arXiv:2210.03629
Tree of Thoughts2023Explore and evaluate branching reasoning paths with searchYao et al., arXiv:2305.10601
RL-trained reasoning models2024–2025Train models with reinforcement learning to produce long internal reasoninge.g. OpenAI o1; DeepSeek-R1, arXiv:2501.12948

Why step-by-step reasoning helps

The mechanism is not fully understood, but a common explanation is computational: generating intermediate steps gives the model more computation per problem, and each step conditions the next, so the model can break a problem into pieces and use earlier results. Self-consistency adds robustness by averaging over several reasoning paths, reducing the impact of any single faulty chain. Tree-based methods go further, generating alternative next steps, evaluating them and pursuing the most promising.

Modern systems combine reasoning with tools and verification. A model working through a maths problem can write and run code to check calculations; one answering a legal or factual question can retrieve sources to ground its steps. This scaffolding catches errors before they compound.

From prompting to trained reasoning models

Since OpenAI’s o1, major labs have released models trained with reinforcement learning to reason before answering. DeepSeek’s R1 report (see our DeepSeek-R1 guide) described training reasoning behaviour largely through RL with verifiable rewards, such as checking whether a maths answer is correct or code passes tests. These models typically expose a setting for reasoning effort, trading cost and latency for accuracy.

This changes how to prompt them. Reasoning models often perform better when given a clear goal, constraints and output format rather than step-by-step instructions, and some providers advise against forcing explicit chain-of-thought prompts with them. For standard chat models, asking for brief reasoning before the final answer still helps on multi-step problems.

Where reasoning helps most

DomainWhy reasoning helpsCaveats
MathematicsProblems decompose into steps; answers are checkableArithmetic slips still occur; verify with code
Coding and debuggingReasoning about logic and causes of bugs improves fixesTests are the real check
PlanningWorking through dependencies and constraints improves coherencePlans can look complete while missing real-world constraints
Analysis of documentsStep-by-step extraction reduces missed detailsMust be grounded in the source text

Limits

Novel problems. Reasoning works best where similar patterns appear in training data. Problems requiring a genuinely new approach remain hard, and step-by-step text does not guarantee insight.

Compounding errors. Each step can introduce a mistake or fabrication that later steps build on, producing a confident but wrong conclusion. The careful structure can create an illusion of reliability.

Cost and latency. Long reasoning uses many more tokens. For simple questions, it adds expense without benefit.

Overthinking. On easy tasks, extended reasoning can sometimes lead models to second-guess correct answers.

Is the visible reasoning faithful?

Research has repeatedly found that stated reasoning does not always reflect what drives a model’s answer. Turpin et al. (2023) showed that models can give plausible explanations while being influenced by biasing features they never mention (arXiv:2305.04388). Treat chains of thought as useful for debugging and building user trust, not as a guaranteed audit trail. This matters for safety research too: monitoring a model’s reasoning is only as reliable as the reasoning is faithful.

Practical guidance

  • For standard chat models, ask for brief reasoning before the final answer on multi-step problems.
  • For reasoning models, specify the goal, constraints and output format; adjust the reasoning-effort setting instead of adding long instructions.
  • Use self-consistency, several samples plus a vote, for high-stakes numerical answers when cost allows.
  • Verify final answers with tools such as code execution, calculators or retrieval rather than trusting the reasoning text.
  • Use low reasoning effort or a non-reasoning model for simple lookups and formatting tasks to save cost.

A worked example

Consider the question: “A shop sells pens at 3 for $2. How much do 12 pens cost?” Asked for an immediate answer, a model might pattern-match to a wrong figure. Prompted to reason, it writes: 12 pens is 4 groups of 3; each group costs $2; 4 × $2 = $8. The intermediate steps make the calculation explicit and easy for a reader, or a tool, to check. For harder problems, the same structure lets you see where a chain went wrong, which is why reasoning output is valuable for debugging even when it is not a perfect window into the model’s internals.

Common questions

Should I always ask models to think step by step? No. It helps on multi-step problems with standard chat models, but adds cost on simple tasks, and reasoning models already do it internally.

Do reasoning models hallucinate less? They tend to do better on structured problems, but they can still produce confident errors, sometimes embedded in long reasoning that looks convincing.

Can I see a reasoning model’s full chain of thought? It depends on the provider. Some show full reasoning, others show summaries or hide it, partly for safety and competitive reasons.

Our analysis

Chain-of-thought is one of the most important practical advances in making language models useful for complex work, and RL-trained reasoning models have made it a built-in capability rather than a prompt trick. But it is best understood as a way to give models more structured computation, not as transparent machine thinking. It is most reliable when answers can be checked and least reliable when a confident chain of plausible steps cannot be verified. The practical rule is simple: let the model reason, then check the result with something other than the model’s own words.