Definition
Chain-of-thought (CoT) prompting is a prompt engineering technique that encourages a large language model to lay out its reasoning step by step before stating a conclusion. Instead of asking "What is the answer?", you ask the model to break the problem into parts, solve each one, and only then answer.
The idea is simple: a language model generates text one token at a time, and every token it writes becomes context for the next. When the model writes out intermediate steps, those steps become working material it can build on. Asked to jump straight to an answer, the model has no room to "show its work" — and multi-step problems suffer.
The research behind it
The technique was named and tested in the 2022 Google Research paper Chain-of-Thought Prompting Elicits Reasoning in Large Language Models by Jason Wei and colleagues. The authors gave models a few worked examples in which each answer was preceded by a written chain of reasoning. On arithmetic, commonsense and symbolic-reasoning benchmarks, this substantially improved performance compared with standard prompting, and the effect grew with model size: it helped very large models far more than small ones. With chain-of-thought examples, the 540-billion-parameter PaLM model reached state-of-the-art accuracy at the time on the GSM8K benchmark of grade-school maths word problems.
A follow-up paper the same year, Large Language Models are Zero-Shot Reasoners by Takeshi Kojima and colleagues, showed that simply adding the phrase "Let's think step by step" — with no worked examples at all — also produced large accuracy gains on reasoning benchmarks. This "zero-shot CoT" is the version most office users apply today.
How it differs from other prompting techniques
| Technique | What you provide | Best for |
|---|---|---|
| Zero-shot prompting | Just the instruction | Simple, well-defined tasks |
| Few-shot prompting | Examples of input and output | Matching a format or style |
| Zero-shot chain-of-thought | Instruction plus "think step by step" | Multi-step reasoning without examples |
| Few-shot chain-of-thought | Examples that include reasoning steps | Complex, repeated reasoning tasks |
When chain-of-thought helps — and when it doesn't
It helps with:
- Calculations with several steps: budgets, pricing, forecasts, unit conversions.
- Logic and rules: checking whether a case meets policy criteria, eligibility checks.
- Planning: breaking a project into tasks, sequencing dependencies.
- Comparisons with multiple criteria: vendor evaluations, option analysis.
- Diagnosing problems: working out why a process or spreadsheet fails.
It adds little to:
- Simple factual lookups or short rewrites.
- Creative writing where there is no single correct answer.
- Tasks where the model lacks the underlying knowledge — reasoning steps cannot replace missing facts, and the model can produce a convincing chain built on a false premise.
Many recent AI assistants include "reasoning" modes that perform this kind of step-by-step thinking internally. Even then, asking for visible steps is useful at work because it lets you check the logic.
A warning: visible reasoning is not proof
A neat chain of reasoning looks trustworthy, but it can still contain an error — a wrong number copied from the input, a misread rule, an AI hallucination presented as a step. Research on the Harvard Business School and BCG experiment found that on a task beyond the AI's capability, consultants using AI were 19 percentage points less likely to reach the right answer than those working without it. Fluent output lowers people's guard. Chain-of-thought's real value at work is that it makes errors easier to find, not that it removes them.
Example
Daniel is a finance business partner asked whether the company should switch from monthly to annual billing for one software vendor. The vendor offers 15% off for paying annually; the team currently has 40 licences at $30 per licence per month, and headcount may fall to 32 within six months.
His first prompt — "Should we switch to annual billing?" — returns a generic paragraph recommending annual billing for the discount.
He rewrites the prompt with chain-of-thought:
"You are a finance analyst. Work through this step by step before giving a recommendation. 1) Calculate the current annual cost on monthly billing. 2) Calculate the annual-billing cost with a 15% discount for 40 licences. 3) Calculate the monthly-billing cost if we drop to 32 licences after six months. 4) Compare scenarios. 5) State the break-even point and a recommendation. Show every calculation."
The model now shows its work: $30 × 40 × 12 = $14,400 on monthly billing; annual billing costs $14,400 × 0.85 = $12,240; if headcount drops to 32 after six months, monthly billing costs (40 × 6 + 32 × 6) × $30 = $12,960. Annual billing is still cheaper by $720 under that scenario — unless licences fall further. Daniel checks each line with a calculator, confirms the numbers, and presents the comparison with the assumptions stated. Because every step is visible, his manager can challenge the headcount assumption directly instead of distrusting the whole answer.
How to use chain-of-thought prompting: step by step
- Confirm the task has steps. If the answer depends on a sequence of calculations, rules or decisions, CoT is a good fit.
- Give full context. State the numbers, rules and constraints explicitly; reasoning is only as good as its inputs.
- Ask for steps before the answer. Use "Work through this step by step, then give your final answer" or "Show your reasoning first."
- Number the steps yourself for important tasks. Supplying the structure (calculate A, then B, then compare) keeps the model on track.
- Add a worked example for repeated tasks. One example with reasoning — few-shot CoT — makes outputs consistent.
- Ask for the final answer in a fixed format. For example: "End with a line beginning 'Recommendation:'."
- Check each step. Recalculate numbers, compare quoted figures with the source, test the logic.
- Ask the model to review itself. "Check your calculations for errors and list any assumptions" often surfaces mistakes, but does not replace your check.
- Save the prompt as a template. Reasoning prompts for recurring analyses are worth reusing.
Reusable templates
- Decision: "List the criteria, score each option against each criterion with a reason, total the scores, then recommend."
- Policy check: "Quote each relevant rule, state whether the case meets it and why, then conclude."
- Calculation: "Write each calculation on its own line with the formula, then the result, then a summary."
Common mistakes
- Using CoT for simple tasks and getting long, padded answers.
- Trusting the reasoning because it looks thorough.
- Leaving key numbers out of the prompt, so the model invents them.
- Not asking for a clear final answer, making the output hard to use.
Key takeaways
- Chain-of-thought prompting asks a model to reason in steps before answering.
- Research from 2022 showed large accuracy gains on reasoning tasks, especially for large models; "Let's think step by step" alone helps.
- It suits calculations, rule checks, planning and comparisons.
- Visible steps make errors easier to catch — but you still have to check them.