Reasoning Effort / Reasoning Levels
How it works
The developer passes an effort level (e.g. low/medium/high) or a thinking-token budget in the API request. The model controls the length of its internal reasoning chain: at low effort it emits a short thinking trace (or skips it), at high effort a longer one, allowing more inference steps, self-verification and solution exploration before the final output. Because thinking tokens are generated autoregressively and are usually billed, higher effort directly increases latency and cost. The exact semantics (discrete levels vs. token budget, whether the thinking is visible) differ across providers.
Problem solved
Reasoning models perform better when they 'think longer', but a fixed, maximal amount of reasoning is slow and costly for simple queries. Reasoning Effort addresses the lack of control over the cost/latency/quality trade-off by putting it in the developer's hands per request.
Key mechanisms
Strengths & limitations
Components
An API request field (e.g. reasoning_effort = low/medium/high) or a thinking-token budget controlling the amount of reasoning.
The model mechanism that generates internal thinking tokens, whose length scales with the requested effort.
Implementation
High effort on simple queries wastes time and tokens with no quality gain.
Thinking tokens are usually billed and lengthen responses; high effort can multiply cost.
Discrete levels vs. token budgets, differing defaults and thinking visibility hurt portability.
Evolution
o1 models introduce test-time compute as a controllable reasoning effort in the API.
Claude exposes a thinking-token budget as a continuous effort control.
Extends the effort scale with a cheapest minimal-reasoning mode.
Gemini exposes discrete thinking levels (low/high) alongside the earlier budget.
Hyperparameters (configurable axes)
Discrete reasoning-effort level.
An alternative, continuous control: the maximum number of tokens allotted to internal thinking (e.g. Anthropic budget_tokens, Gemini thinking budget).
Computational complexity
Time complexity: O(b).
Compute bottleneck
The internal reasoning chain is generated autoregressively token by token, making latency the main constraint at high effort.
Execution paradigm
The amount of compute depends on the requested effort (and, with adaptive thinking, on query difficulty).
Parallelism
Longer thinking means a longer token sequence generated serially; parallelism applies more to 'parallel test-time compute' (many runs at once) than to a single chain.
Hardware requirements
Generating long thinking traces is intensive LLM decoding on GPUs.
Reasoning models are also served on TPUs (e.g. Gemini).
It is a control parameter, independent of specific hardware.