Wei et al. (2022): Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

1. Introduction & Context

Large language models have demonstrated remarkable fluency and factual recall, yet their ability to perform structured reasoning has historically lagged. Prior to Wei et al. (2022), researchers relied on zero-shot prompting or fine-tuning to extract reasoning capabilities, often yielding inconsistent results on tasks requiring multi-step logic.

The 2022 paper emerged during a period of rapid scaling in transformer-based models. The authors hypothesized that the latent reasoning abilities of LLMs could be activated not through architectural changes or training data expansion, but through carefully structured prompt design that mirrors human step-by-step problem solving.

2. Methodology: Chain-of-Thought Prompting

Chain-of-Thought (CoT) prompting involves appending a series of intermediate reasoning steps to few-shot examples in the prompt. For example, instead of asking "Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 balls. How many balls does he have now? A: 11", the prompt includes the explicit calculation pathway:

By exposing the model to this format, Wei et al. demonstrated that the model learns to generate its own intermediate steps when faced with novel questions, effectively simulating a reasoning trace before producing a final answer.

3. Key Findings & Benchmark Results

The study evaluated CoT prompting across multiple benchmarks, including:

• GSM8K (Grade School Math): Accuracy improved from 18% (zero-shot) to 78.7% with CoT.
• SVAMP (Word Problems): Significant gains in arithmetic reasoning.
• StrategyQA (Multi-step commonsense): CoT outperformed standard prompting by over 20% absolute.
• EntailmentBank (Natural language inference): Improved traceability of logical connections.

Crucially, the technique showed a strong scaling effect: larger models (e.g., PaLM 540B) benefited disproportionately from CoT prompting compared to smaller variants, suggesting that reasoning capabilities emerge at scale when properly triggered.

4. Implications for AI Research

The introduction of Chain-of-Thought prompting fundamentally shifted how researchers interact with LLMs. It demonstrated that:

1. Reasoning can be elicited rather than trained, reducing the need for expensive fine-tuning on synthetic reasoning datasets.
2. Model transparency improves when forced to output intermediate steps, enabling better error analysis.
3. Prompt engineering became a first-class research discipline, bridging NLP, cognitive science, and usability design.

The technique also laid groundwork for subsequent innovations like Self-Consistency, Tree-of-Thoughts, and ReAct (Reasoning + Acting).

5. Limitations & Criticisms

Despite its impact, CoT prompting has documented limitations:

• Brittleness: Performance drops significantly on tasks requiring precise formatting or domain-specific notation not seen in the prompt examples.
• Verification Gap: Generated reasoning steps are not guaranteed to be logically sound; models can fabricate plausible but incorrect intermediate steps ("reasoning hallucination").
• Token Overhead: Generating full reasoning traces increases inference cost and latency, making it less viable for production systems with strict constraints.

Subsequent research has focused on automated verification of reasoning traces and hybrid approaches combining neural generation with symbolic validation.

6. Legacy & Subsequent Work

Wei et al. (2022) has been cited thousands of times and is considered a cornerstone of modern LLM prompting research. The concept has evolved into structured reasoning frameworks, agentic workflows, and neuro-symbolic hybrid systems. Aevum Encyclopedia tracks ongoing developments in this lineage under the Reasoning Architectures and Prompt Engineering categories.