Zero-shot-CoT
How it works
The method runs in two sequential prompting stages on the same model: 1) Reasoning extraction: the trigger phrase 'Let's think step by step' is appended to the question, and the model autoregressively generates an explicit chain of intermediate reasoning steps. 2) Answer extraction: the generated chain is concatenated back into the prompt together with an answer-forcing phrase (e.g. 'Therefore, the answer (arabic numerals) is'), and the model produces the final, parseable answer. The whole procedure uses one fixed template across all tasks — no exemplars and no task-specific tuning. Output quality increases with model scale.
Problem solved
Classic few-shot Chain-of-Thought requires hand-crafted, task-specific reasoning exemplars — this is costly, brittle, and generalizes poorly to new task types. Zero-shot-CoT removes the need for exemplars entirely: one task-agnostic trigger phrase unlocks multi-step reasoning on any task, revealing the latent reasoning abilities of large language models.
Components
A universal, task-agnostic phrase appended to the question that induces the model to generate an explicit reasoning chain. Its exact wording materially affects performance.
Official
First model pass: question + trigger phrase as input, a chain of intermediate reasoning steps as output.
Second model pass: question + generated chain + answer-forcing phrase (e.g. 'Therefore, the answer is') as input, a parsed final answer as output.
Official
Implementation
Without the second extraction stage the answer is buried in the reasoning text; naive regex parsing often fails across formats.
The effect is emergent — small models do not produce useful reasoning and may lose accuracy.
Small changes to the trigger phrase can noticeably change results.
Evolution
Wei et al. show that step-by-step reasoning exemplars improve LLM reasoning — but require hand-crafted demonstrations per task.
Kojima et al. prove that a single task-agnostic phrase elicits reasoning with no exemplars, dramatically improving zero-shot results (MultiArith 17.7%→78.7%, GSM8K 10.4%→40.7%).
Sampling many reasoning chains and majority voting further boosts accuracy — composes with Zero-shot-CoT.
Zhang et al. use Zero-shot-CoT to auto-generate reasoning exemplars, building few-shot CoT without manual labor.
Hyperparameters (configurable axes)
The exact wording of the reasoning-inducing phrase. It has a critical impact on performance.
The stage-2 phrase that forces the final answer in a parseable format.
Parameter count of the base model. The Zero-shot-CoT benefit is emergent — it appears only at large scale.
Sampling method (greedy vs sampling). Greedy yields a single chain; sampling enables combining with Self-Consistency.
Token budget for the stage-1 reasoning chain — a trade-off between reasoning completeness and latency/cost.
Computational complexity
Time complexity: O(T_r + T_a). Space complexity: O(n).
Compute bottleneck
The bottleneck is token-by-token generation of the long reasoning chain plus two sequential model passes, which increase end-to-end latency.
Execution paradigm
Zero-shot-CoT is a prompting layer over the base model — it does not change the model's internal execution; all paths of the dense model remain active.
A prompting technique that runs on any dense language model without routing.
Parallelism
Multiple independent queries can be parallelized (batched), but not the steps within a single query.
Hardware requirements
Zero-shot-CoT is a prompting layer — it runs on any hardware capable of serving the base model.
In practice the large base models run on tensor-core GPUs; longer reasoning chains increase inference-throughput demand.