The developer passes an effort level (e.g. low/medium/high) or a thinking-token budget in the API request. The model controls the length of its internal reasoning chain: at low effort it emits a short thinking trace (or skips it), at high effort a longer one, allowing more inference steps, self-verification and solution exploration before the final output. Because thinking tokens are generated autoregressively and are usually billed, higher effort directly increases latency and cost. The exact semantics (discrete levels vs. token budget, whether the thinking is visible) differ across providers.
Reasoning models perform better when they 'think longer', but a fixed, maximal amount of reasoning is slow and costly for simple queries. Reasoning Effort addresses the lack of control over the cost/latency/quality trade-off by putting it in the developer's hands per request.
An API request field (e.g. reasoning_effort = low/medium/high) or a thinking-token budget controlling the amount of reasoning.
The model mechanism that generates internal thinking tokens, whose length scales with the requested effort.
High effort on simple queries wastes time and tokens with no quality gain.
Thinking tokens are usually billed and lengthen responses; high effort can multiply cost.
Discrete levels vs. token budgets, differing defaults and thinking visibility hurt portability.
o1 models introduce test-time compute as a controllable reasoning effort in the API.
Claude exposes a thinking-token budget as a continuous effort control.
Extends the effort scale with a cheapest minimal-reasoning mode.
Gemini exposes discrete thinking levels (low/high) alongside the earlier budget.
Time complexity: O(b).
The internal reasoning chain is generated autoregressively token by token, making latency the main constraint at high effort.
Discrete reasoning-effort level.
An alternative, continuous control: the maximum number of tokens allotted to internal thinking (e.g. Anthropic budget_tokens, Gemini thinking budget).
The amount of compute depends on the requested effort (and, with adaptive thinking, on query difficulty).
Longer thinking means a longer token sequence generated serially; parallelism applies more to 'parallel test-time compute' (many runs at once) than to a single chain.
Generating long thinking traces is intensive LLM decoding on GPUs.
Reasoning models are also served on TPUs (e.g. Gemini).
It is a control parameter, independent of specific hardware.
