Which Prompting Technique Should You Use? CoT vs Self-Consistency vs ReAct vs Tree of Thoughts

Most "prompt engineering cheat sheet" posts list technique names side by side and leave you to guess which one actually fixes your problem. The techniques below solve different failure modes, so the right question isn't "which is best" — it's "which failure am I seeing." Each section names the failure, the fix, the paper it comes from, and what the fix costs in extra model calls.

Start here: match the failure to the fix

Chain-of-thought: the default fix for multi-step problems

Wei et al. (2022), "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models," showed that adding a handful of worked examples with explicit reasoning steps — rather than just question-answer pairs — sharply improves accuracy on arithmetic, commonsense, and symbolic reasoning tasks. Kojima et al. (2022), "Large Language Models are Zero-Shot Reasoners," found you don't even need worked examples: appending the instruction "Let's think step by step" before the model answers produces most of the same benefit with zero labeled demonstrations.

Worked example — a bakery starts the day with 34 loaves, sells 19 by 10 a.m., receives a delivery of 4 trays of 6 loaves each, then sells 5 more for a wedding order. Asked directly for the final count, a model can skip a step and answer 39 (delivery counted, final sale forgotten). Asked to work through it step by step: "34 − 19 = 15. Delivery adds 4 × 6 = 24, so 15 + 24 = 39. The wedding order removes 5, so 39 − 5 = 34." Same model, same facts — writing out the intermediate state stops the last step from getting dropped.

CoT is the right default whenever a task has more than one dependent step. It is not a fix for missing knowledge (the model can reason cleanly from a wrong fact) and it is not free of quirks: Sclar et al. (2023), "Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design," found that purely cosmetic formatting choices — separators, spacing, example order — can swing accuracy by double-digit percentage points even when the wording is unchanged, so don't assume one prompt format generalizes untested to a new task.

Self-consistency: when one reasoning path isn't reliable enough

Wang et al. (2022), "Self-Consistency Improves Chain of Thought Reasoning in Language Models," propose sampling several independent chain-of-thought completions for the same question (instead of decoding one greedily) and taking a majority vote over the final answers. On PaLM-540B, this lifted GSM8K accuracy by +17.9 points, SVAMP by +11.0, and AQuA by +12.2 over standard greedy CoT, using up to 40 sampled paths per question in the paper's main results.

Continuing the bakery example: sample the CoT prompt five times independently and you might get final answers 34, 34, 39, 34, 28 — arithmetic slips in two of the five runs, but 34 wins the vote 3-to-2. That's the mechanism: individual runs still make mistakes, but the mistakes rarely agree with each other, while the correct path keeps recurring.

Self-consistency only works when the task produces a checkable, poolable final answer (a number, a label, a short extractable string) — it has nothing to vote on for open-ended writing. It's also the most expensive fix on this list per point of accuracy gained, since it multiplies your call count directly; reach for it only after confirming plain CoT is inconsistent across repeated runs, not as a first move.

ReAct: when the answer depends on a fact, not just reasoning

Yao et al. (2022), "ReAct: Synergizing Reasoning and Acting in Language Models," interleave short reasoning traces ("Thought") with tool calls ("Action") and the tool's result ("Observation"), letting the model revise its plan based on what it actually finds rather than reasoning purely from memory.

Worked example — "Who directed the film that won the Academy Award for Best Picture at the 1994 ceremony, and in what year was that director born?" A pure CoT run answers from parametric memory and risks naming the wrong film or misremembering a birth year. A ReAct run instead does: Thought: I need the 1994 Best Picture winner. Action: Search("1994 Academy Award Best Picture winner"). Observation: Schindler's List. Thought: I need its director. Action: Search("director of Schindler's List"). Observation: Steven Spielberg. Thought: I need his birth year. Action: Search("Steven Spielberg birth date"). Observation: born December 18, 1946. Final answer: 1946 — each hop is grounded in a retrieved fact instead of a guess.

Use ReAct whenever the honest failure mode is "the model might not know this" or "this changes over time," not "the model can't reason through this." Bolting search onto a task that's purely a reasoning puzzle (like Game of 24 below) wastes tool calls and doesn't help.

Tree of Thoughts: when an early move can be a dead end

CoT and self-consistency both commit to one path per sample and never look back mid-answer. Yao et al. (2023), "Tree of Thoughts: Deliberate Problem Solving with Large Language Models," instead let the model generate several candidate "next thoughts" at each step, score how promising each one looks, keep only the best few (a beam), and backtrack out of branches that stop looking promising — closer to a search algorithm than a single sampled continuation.

The paper's headline result is the Game of 24 (combine four given numbers with +, −, ×, ÷ to reach exactly 24): GPT-4 with standard chain-of-thought prompting solved only 4% of test instances in one sampled attempt each, while GPT-4 with Tree of Thoughts (breadth b=5, three value samples per candidate thought) solved 74% — because a bad first move that looks fine for a step or two can be abandoned before it wastes the whole solution. That reliability isn't free: the paper reports around 70 tree nodes visited per problem, each requiring its own generation and evaluation call, putting total compute at over 100 times a single CoT sample for the same question.

Reach for ToT only when getting the early steps wrong is genuinely common and costly to discover late — planning, multi-step puzzles, agentic task decomposition. For most single-pass reasoning questions, CoT or self-consistency gets you most of the accuracy at a fraction of the cost.

One more fix worth knowing: PAL, for when reasoning is fine but arithmetic isn't

Sometimes the CoT plan is entirely correct and the model still gets the final number wrong because it's doing multi-digit arithmetic in its head. Gao et al. (2022), "PAL: Program-Aided Language Models," have the model write out the reasoning as executable code (so a calculator or interpreter does the arithmetic) instead of computing the final answer itself in natural language. It's a one-call fix, not a search or sampling strategy, so it composes cheaply with CoT rather than competing with self-consistency or ToT for the same budget.

Putting it together

Start with plain CoT. If a task has more than one dependent step, that's usually enough. Add self-consistency only once you've confirmed CoT gives inconsistent answers across repeated runs and the task has a poolable final answer. Add ReAct when the failure is a wrong or missing fact, not a broken reasoning chain. Reserve Tree of Thoughts for planning-shaped problems where an early wrong turn is expensive to discover late — it is the most powerful and the most expensive tool here by a wide margin. And if the plan is right but the arithmetic isn't, reach for PAL before reaching for anything more expensive.

For more on the papers behind these techniques — plus zero-shot vs. few-shot prompting, prompt injection, prompt caching, and dozens of other named techniques with citations — try the AI engineering prompting practice quiz.

Source: Wei et al., "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" (arXiv:2201.11903); Kojima et al., "Large Language Models are Zero-Shot Reasoners" (arXiv:2205.11916); Wang et al., "Self-Consistency Improves Chain of Thought Reasoning in Language Models" (arXiv:2203.11171); Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models" (arXiv:2210.03629); Yao et al., "Tree of Thoughts: Deliberate Problem Solving with Large Language Models" (arXiv:2305.10601); Gao et al., "PAL: Program-Aided Language Models" (arXiv:2211.10435); Sclar et al., "Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design" (arXiv:2310.11324).

Drill Prompt Engineering practice questions →