Reasoning Tokens: API Semantics, Billing, and Visibility

Abstract. Modern reasoning models from OpenAI, Anthropic, and Google generate substantial hidden inference work before producing a visible answer. The vendors expose this work through related but non-interchangeable interfaces — OpenAI's reasoning items, Anthropic's thinking blocks, Google's thought signatures — each with distinct semantics for control, visibility, billing, and replay. This article maps those interfaces to a common conceptual frame, documents the billing and continuity rules that practitioners actually have to honor, and argues that "reasoning tokens" should be treated as billable opaque state with stochastic cost rather than as a faithful audit trail of model cognition.

Coverage note: verified through May 2026 against current OpenAI Responses API, Anthropic Claude API, and Google Gemini API documentation. Model defaults and parameter shapes have changed within the past year and should be re-checked before quoting.

What "reasoning tokens" refers to

The phrase "reasoning tokens" entered practitioner usage with OpenAI's o1 release in late 2024 and has since been retrofitted onto roughly parallel mechanisms in other vendors' APIs. The shared idea is that a reasoning model can spend an extended budget of generated tokens on internal deliberation before, or interleaved with, the tokens that constitute its visible response. That deliberation is hidden from the client by default; some vendors expose a summary, some expose an encrypted continuity blob, and none expose the raw token sequence in a stable, auditable form.

It is tempting to treat "reasoning tokens" as a portable cross-vendor primitive. It is not. A reasoning token in OpenAI's accounting, a Claude thinking block, and a Gemini thought signature are not interchangeable artifacts. They differ on at least four axes:

Granularity — whether the object is a token count, a structured block, or an opaque state handle.
Visibility — whether the client sees raw text, a summary, an encrypted blob, or only a usage field.
Billing surface — whether the work is billed as output tokens, output-plus-thinking tokens, or with separate cache and continuation pricing.
Continuity contract — whether the client must round-trip a specific artifact to preserve reasoning state across tool calls or turns.

Where this article uses the phrase "reasoning tokens" without qualification, it refers to the practitioner-level concept of generated, billable, partially hidden inference state. Vendor-specific terms (reasoning items, thinking blocks, thought signatures) are used wherever the difference matters.

The conceptual frame for the rest of the article is three orthogonal questions. Each vendor's design answers them differently.

Axis	Question	Why it matters
Control	How does the client request more or less reasoning?	Determines both quality and cost envelope per call.
Visibility	What part of the reasoning, if any, comes back to the client?	Determines what can be logged, evaluated, or audited.
Continuity	What must the client send back to preserve reasoning across turns or tool calls?	Determines correctness in multi-step agentic workflows.

Billing semantics cut across all three. The interaction is what makes the topic operationally awkward, and why "reasoning tokens" gets discussed as a single thing even though the underlying primitives are not.

Vendor semantics

OpenAI: hidden reasoning, optional summaries, replayable items

OpenAI's reasoning models (the o-series and reasoning-capable GPT-5 variants) emit hidden reasoning tokens that the client never sees in raw form. The current Responses API reasoning guide describes them as internal generated tokens that the model uses to "think through" the problem; the API exposes only the count in usage.output_tokens_details.reasoning_tokens, plus optional natural-language summaries when requested.

The control surface is the nested reasoning object on the request:

{
  "model": "o3",
  "reasoning": {
    "effort": "medium",
    "summary": "auto"
  },
  "input": "..."
}

reasoning.effort takes values like minimal, low, medium, and high, with availability varying by model. reasoning.summary controls whether a natural-language summary of the reasoning trace is returned, and how detailed it is. The older top-level generate_summary parameter is deprecated. Some SDK wrappers and config files expose flattened names like reasoning_effort and reasoning_summary; those are wrapper conventions, not the documented Responses API shape.

Three properties of OpenAI's design have direct operational consequences:

Reasoning tokens are billed as output tokens. They count against the same per-token output price as visible response tokens, and they accrue into usage.output_tokens (broken out under output_tokens_details.reasoning_tokens for inspection).
Reasoning tokens count against max_output_tokens. A call with a tight cap can spend the entire budget on hidden reasoning and return an empty or truncated visible response. The Responses API marks such responses as incomplete with incomplete_details.reason = "max_output_tokens". OpenAI's reasoning best-practices guide explicitly recommends reserving headroom — a common rule of thumb is at least 25,000 reserved output tokens for medium or high effort on complex tasks — and then sizing down with telemetry.
Reasoning state has to be carried forward deliberately. When a reasoning model is used inside a tool-call loop, the model's internal reasoning needs to be returned to the next turn for it to "remember" what it was doing. The Responses API offers two mechanisms: previous_response_id (stateful, server-side), or explicit replay of reasoning items in the input array. For zero-data-retention (ZDR) accounts where server-side state is not retained, reasoning.encrypted_content provides an opaque blob that the client stores and replays. Dropping these items between tool calls causes the model to re-derive context, which raises both reasoning-token cost and the risk of inconsistent decisions.

The Chat Completions API on the same models still works but is stateless and does not preserve reasoning context across calls. OpenAI's documentation recommends Responses for any multi-step agentic use precisely because Chat Completions can "use more reasoning tokens" on otherwise identical tasks. That cost gap is itself an under-appreciated design signal: the stateful API is being positioned as the cost-correct default.

Anthropic: thinking blocks, redacted blocks, signed continuity

Anthropic's extended thinking documentation describes a different surface. When extended thinking is enabled, Claude returns one or more thinking content blocks in the response, interleaved with text blocks, before producing the final answer. The thinking blocks are first-class items in the content array, not a separate stream or usage field.

The control parameter is on the request body:

{
  "model": "claude-opus-4-7",
  "thinking": {
    "type": "enabled",
    "budget_tokens": 16000
  },
  "messages": [...]
}

budget_tokens is a hint, not a hard cap on the visible response — it bounds how many tokens Claude will spend on thinking before producing the answer. Model-class behavior varies sharply across the Claude 4 generation and is worth tracking against the official docs:

Earlier Claude 4 models supported explicit budget_tokens and returned full thinking blocks for client inspection.
Newer Opus-class models in the 4.6–4.7 line have moved toward adaptive thinking, with budgets either deprecated or reinterpreted. The docs explicitly state that Opus 4.7 no longer accepts manual budget_tokens and instead determines effort adaptively.
Sonnet- and Haiku-class models retain explicit budgets in many configurations.

Visibility also varies. Anthropic documents at least four states a thinking block can be in:

State	Content visible to client	When
Full thinking	Raw thinking text plus signature	Older Claude 4 generations and some configurations
Summarized thinking	Shorter natural-language summary plus signature	Common on newer models; `display` configuration may influence
Omitted thinking	Only a signature, no text	When the platform chooses not to surface the trace
Redacted thinking	`redacted_thinking` block; opaque payload	Triggered by safety classifiers on the thinking trace itself

Two facts about Anthropic's design are easy to get wrong:

Billed thinking tokens can substantially exceed visible thinking tokens. Anthropic states that customers are billed for the full thinking process, not the visible summary. A response that returns a 200-token thinking summary may have been generated against thousands of billed thinking tokens. Reconciling invoice spend against visible output is therefore not possible from the response payload alone; the practitioner must trust the usage fields and treat the visible summary as observability, not accounting.
Thinking blocks must be round-tripped unmodified during tool use. The thinking and redacted_thinking blocks carry cryptographic signatures. When the API is invoked with tool_use and the assistant returns a thinking block followed by a tool call, the next user turn (carrying the tool result) must include the original thinking block verbatim. Modifying, reordering, or stripping it will cause Claude to reject the continuation or silently lose state. This is enforced because the signature binds the thinking content to a specific reasoning context; without the original block, the model has no faithful continuation.

Prompt-cache interaction is a third trap. The thinking parameters are part of the cache key. Changing budget_tokens, toggling thinking on or off, or upgrading to a model with different defaults invalidates cache breakpoints that would otherwise have shared prefix work. Stable thinking parameters and stable system prompts are prerequisites for cache hit rates worth talking about.

Google: thought signatures and budgeted thinking

Google's Gemini API takes a third approach, documented in the Gemini thinking guide and the thought signatures guide.

Control is exposed through one of two parameters, depending on model generation:

Gemini 2.5 models use thinkingBudget (integer), with -1 denoting unlimited (model-determined) thinking.
Gemini 3 models use thinkingLevel (low, medium, high), a categorical analogue to OpenAI's reasoning.effort.

includeThoughts: true requests a natural-language summary of thinking, returned as content parts where part.thought == true. The raw thought stream is not exposed.

Continuity is the part of Google's design that most often surprises practitioners. Gemini does not return thinking text by default; what it does return — particularly in function-calling and multi-turn flows — is a thoughtSignature: an opaque, encrypted blob that encodes enough of the reasoning state for the model to resume. The signature is attached to specific content parts and must be returned to the API unchanged on the next turn.

Important properties of thoughtSignature:

It is not human-readable. It is base64-encoded ciphertext, not a transcript. Treating it as a debug surface is a category error.
It is mandatory for Gemini 3 function calling. The function-calling docs state that the model will return 400 errors if a function-result turn omits the signature that was attached to the corresponding function-call part. Earlier Gemini versions were more permissive, which has made retrofits to Gemini 3 a recurring source of regressions.
It is billed. Returning the signature on the next turn counts those bytes as input tokens. The thoughtsTokenCount field in the usage metadata reflects thinking tokens generated on the current call; once a signature is replayed, its size is reflected in input token counts.
It interacts with stateless deployments. Vertex AI and the Gemini API have slightly different idioms for state retention, and the signature is the mechanism that makes stateless flows correct without the client ever seeing the underlying reasoning.

Pricing is documented as the sum of input tokens, output tokens, and thinking tokens, with thinking tokens reported separately in usage_metadata.thoughtsTokenCount. Some pricing pages collapse this into a single output-token line item; the usage payload is the authoritative reference for what was actually billed.

Comparison

Property	OpenAI (Responses API)	Anthropic (Claude API)	Google (Gemini API)
Control parameter	`reasoning.effort` (`minimal`/`low`/`medium`/`high`)	`thinking.budget_tokens` (integer); adaptive on newest models	`thinkingBudget` (2.5) or `thinkingLevel` (3)
Raw reasoning visible to client	No	Sometimes (model-dependent: full / summary / omitted / redacted)	No
Summary available	Yes, via `reasoning.summary`	Yes, on newer models	Yes, via `includeThoughts: true`
Continuity artifact	Reasoning items, `previous_response_id`, or `reasoning.encrypted_content`	`thinking` and `redacted_thinking` blocks (with signatures), round-tripped verbatim	`thoughtSignature` blobs on content parts, replayed on next turn
Billing	Reasoning tokens billed as output; appear in `output_tokens_details.reasoning_tokens`	Full thinking process billed (not just visible summary); cache invalidated by thinking changes	Output plus thinking tokens; `thoughtsTokenCount` in usage; replayed signatures count as input
Counts against output cap	Yes (`max_output_tokens` covers reasoning + visible)	Effectively yes (`budget_tokens` bounds thinking; total output enforced separately)	Yes (configurable; `thinkingBudget` is a separate axis)
Mandatory replay	For multi-step tool use; ZDR requires `encrypted_content`	For tool-use turns; signatures bind blocks to context	For Gemini 3 function-call continuations

The table is useful as a cheat sheet, but the rows are not equivalent objects. An OpenAI reasoning token is a unit of generated text the client never sees; a Claude thinking block is a structured content item that may or may not contain visible text; a Gemini thought signature is an encrypted state handle that contains no readable content at all. The cross-vendor concept is "billable hidden inference state with vendor-specific continuity rules," not "the model's thoughts."

Billing semantics

The billing model for reasoning work is more subtle than "hidden output tokens cost the same as visible output tokens." Three patterns recur and three traps follow.

Reasoning is output-side spend

Across all three vendors, reasoning work is billed on the output side rather than the input side. OpenAI counts reasoning tokens directly into usage.output_tokens. Anthropic's billed output exceeds the visible-text token count because billed output includes the full thinking process. Google reports thoughtsTokenCount as a separate field but prices it under the output rate. This means the cheap-input/expensive-output asymmetry that practitioners learned from non-reasoning models is amplified rather than reversed: reasoning models do more work per call, and that work shows up as output.

The pricing implication is that an order-of-magnitude increase in output per query is not unusual when moving from a non-reasoning model to a high-effort reasoning model on the same task. Practitioners who size budgets from a non-reasoning baseline routinely underestimate by 3–10x.

Visible output is not a reliable proxy for billed work

For Anthropic in particular, the visible response can be dramatically smaller than the billed work. A summarized thinking block may surface 200 tokens of natural language while billing thousands of underlying thinking tokens. The same dynamic appears at OpenAI: a five-sentence response from a reasoning model may have been backed by tens of thousands of reasoning tokens that never appear in the assistant message.

Two consequences:

Cost telemetry must read usage fields directly. Counting tokens in the response body is not a substitute. The fields to log are usage.output_tokens_details.reasoning_tokens (OpenAI), the response's usage block compared against visible text (Anthropic), and usage_metadata.thoughtsTokenCount (Google).
Cost evaluations must include reasoning settings as a dimension. A benchmark that pits Claude against GPT against Gemini "on the same task" without recording reasoning.effort, thinking.budget_tokens, or thinkingLevel is measuring four variables and reporting one number. See Cross-Vendor Model Evaluation for the broader pitfall.

`max_output_tokens` is a shared budget

OpenAI's max_output_tokens covers reasoning plus visible output. A request can hit the cap during reasoning and return no useful text. The response is marked incomplete and the bill still arrives. Reserving headroom is therefore not a hedge but a correctness requirement; OpenAI's guidance suggests reserving at least 25,000 tokens for medium/high effort on hard tasks. Anthropic's budget_tokens operates the same way for thinking specifically and is paired with an independent visible-output limit. Google's thinkingBudget is separate from the visible-output limit but interacts with it through the overall context window.

The failure mode worth naming is "spent reasoning, no visible answer." It is silent in dashboards that track only error rates and visible-output counts. Healthy telemetry separates four states: success with visible output, retry, hard error, and reasoning-truncation (incomplete due to budget exhaustion).

Visibility is not auditability

The most consequential interpretive error in practitioner discussions is treating exposed reasoning — a summary, a thinking block, a thought stream — as a faithful trace of the computation that produced the answer. The empirical literature does not support that interpretation.

Wei et al. (2022) established that chain-of-thought prompting improves performance on a range of reasoning benchmarks, particularly in larger models. That finding is robust and informs why vendors invest in reasoning at all. It does not, however, establish that the chain-of-thought is causally responsible for the answer in the way an explanation is causally responsible for a conclusion.

Turpin et al. (2023) demonstrated that chain-of-thought explanations can systematically misrepresent the actual drivers of model behavior. In their experiments, biasing features (such as the position of the correct answer in a multiple-choice prompt) shifted model answers substantially while the CoT explanations made no reference to the biasing feature and presented post-hoc justifications instead.

Lanham et al. (2023) showed that CoT faithfulness — the degree to which the visible reasoning actually controls the final answer — varies sharply by task and model. Counterintuitively, faithfulness can decrease with scale on some tasks: more capable models become more able to produce correct answers without their CoT actually being load-bearing.

Anthropic's own faithfulness research (Anthropic 2025) reported that Claude 3.7 Sonnet referenced inserted hints in its CoT only about a quarter of the time on average, even when those hints clearly influenced the final answer. The accompanying analysis frames CoT as a useful but unreliable monitoring channel, not an audit log.

OpenAI's chain-of-thought monitoring research makes a parallel claim from the safety side: CoT monitoring can catch certain misbehaviors that final-output monitoring would miss, but directly optimizing on CoT tends to teach models to hide intent. The implication is that the visibility of reasoning is most useful when treated as an unoptimized side channel — useful for detection precisely because the model is not being trained to manage what appears there. Once it becomes a product surface, that property erodes. The follow-up evaluating chain-of-thought monitorability study extends the argument empirically.

The synthesis worth holding is:

Visible reasoning is a useful debugging and monitoring signal.
Visible reasoning is not a faithful explanation of why the model answered as it did.
Encrypted continuity artifacts (OpenAI's encrypted_content, Google's thoughtSignature, Anthropic's signed blocks) are not transcripts at all — they are state handles for correctness in multi-step flows.

A "show me the reasoning" UX is reasonable for end users when framed as informational. It is not a substitute for evaluations that score behavior against expected outputs. See Faithfulness of Chain-of-Thought for a deeper treatment of the empirical literature.

Operational consequences for orchestrators

The semantics above translate into concrete changes in how an orchestrator should be built. Five operational consequences recur across deployments.

Token estimation becomes distributional

For non-reasoning models, a reasonable estimator for cost-per-call is "prompt size in input tokens, expected response size in output tokens, both knowable in advance within a tight band." For reasoning models, the hidden-reasoning component is the dominant variance term and is not reliably knowable per call.

The right replacement is empirical telemetry: for each task class and each provider, log the per-call distribution of reasoning_tokens (OpenAI), billed-output-minus-visible-text (Anthropic), and thoughtsTokenCount (Google). Budget against P50, P95, and P99 of that distribution rather than against a single estimate. The P99 is what determines whether max_output_tokens is set high enough to avoid truncation; the P50 is what determines unit economics.

For a concrete operational rule: the orchestrator should not promise a cost ceiling without knowing the P99 of hidden reasoning for the task class. If telemetry is unavailable, the first deployment should treat any hard cost ceiling as provisional until P95 stabilizes.

Retries can lose reasoning

Naive retry logic — replay the same prompt on transient errors — can be expensive in two ways with reasoning models. First, each retry pays full reasoning cost again; the model does not "remember" its previous reasoning across separate API calls. Second, in tool-use loops, dropping the reasoning artifacts on retry can cause the model to re-derive context, which inflates reasoning cost on the retry and can change the action sequence in unstable ways.

Defensive patterns:

Preserve provider-specific reasoning state across retries within a logical operation. OpenAI's previous_response_id or replayed reasoning items, Anthropic's signed thinking blocks, and Google's thoughtSignature blobs must each be carried forward verbatim where the operation is conceptually "the same call, retried."
Distinguish transient transport errors from semantic failures. A 429 should be retried with state preserved; a 400 indicating an invalidated signature is a different failure mode and requires re-deriving reasoning rather than replaying.
Cap retry-driven reasoning amplification. Three retries on a high-effort reasoning task can quadruple the reasoning cost if each retry regenerates the trace. Budget retries against reasoning telemetry, not just request counts.

Cache invalidation differs by axis

Prompt caching works at the token-prefix level, and reasoning parameters interact with the cache key. The interactions are vendor-specific:

On Anthropic, thinking.budget_tokens and thinking.type are part of the cache breakpoint. Toggling thinking on or off, or moving from one budget to another, invalidates prefixes that would otherwise hit the cache. For a workload that wants prompt caching, the thinking configuration must be stable.
On OpenAI Responses, replayed reasoning items become part of the input on subsequent calls. Their token count contributes to cache eligibility through normal prefix matching, but their replay also pins certain semantic state to the current request. Restructuring reasoning replay across calls changes the effective prefix.
On Google, the thoughtSignature is bound to the content parts it is attached to. Reordering parts or modifying surrounding text can break the implicit cache that the signature represents at the model side.

The practical rule is to treat the cache key as a tuple of (prompt prefix, tool definitions, reasoning configuration, replayed reasoning state). Changing any of these axes invalidates the cache. See Prompt Caching Strategies for the broader treatment.

Budgets must be three-layered

A single max_tokens cap is not enough for reasoning workloads. Three layers cover the real cost surface:

A reasoning budget (reasoning.effort, thinking.budget_tokens, or thinkingLevel) that controls how much hidden work the model will do per call.
A visible-output budget (max_output_tokens or equivalent), set with enough headroom above the reasoning budget that the call cannot end as incomplete-due-to-truncation.
A call-level cost ceiling enforced in the orchestrator from observed telemetry — kill or downgrade the call if cumulative billed output exceeds the P99 envelope for the task class.

Many production orchestrators set only the second layer and discover the first and third the hard way. A common failure pattern is to set max_output_tokens at "expected response size plus small margin," not realizing that on a reasoning model this can be exhausted by hidden reasoning before a single visible token appears.

Reasoning artifacts are sensitive operational data

Thinking blocks (summarized or otherwise), reasoning summaries, redacted thinking payloads, encrypted reasoning content, and thought signatures are all things a careful security review will treat as sensitive. Reasoning text can contain user PII, confidential prompts, or internal heuristics; encrypted blobs are still data subject to retention controls.

Three implications:

Logging policies should redact thinking content by default, surfacing only usage counts and request metadata. A diagnostic mode that captures full thinking text should be guarded and time-limited.
Retention rules need to apply to continuity artifacts as well as primary content. A thoughtSignature stored to enable a multi-turn retry inherits the retention policy of the underlying conversation.
Cross-account or cross-tenant replay of reasoning artifacts is a vector worth red-teaming. OpenAI's encrypted_content and Google's thoughtSignature are scoped server-side; misrouting them is unlikely to break the encryption but is well within the failure modes of a poorly partitioned orchestrator.

The API-design debate

The vendors have not converged because the design choices reflect unresolved trade-offs, not just engineering preference. Five tensions structure the debate.

Visibility versus optimizability

A visible chain-of-thought is monitorable. OpenAI's monitorability research argues that CoT visibility — for safety teams, not necessarily for end users — improves detection of certain classes of misaligned behavior. But the same research warns that once CoT becomes a target, models learn to manage what appears there. The strongest argument for keeping raw reasoning hidden from clients is not user safety; it is that an unoptimized CoT is more useful for vendor-side monitoring than an optimized one.

The corollary is that the asymmetry between vendor and client visibility is a feature, not a bug. The vendor sees the raw trace; the client sees a summary or signature. Practitioners who argue for "full visibility into reasoning" are implicitly asking for the design that makes the channel less useful for safety monitoring.

Faithfulness versus utility

A faithful chain-of-thought is one that causally controls the answer. A useful chain-of-thought is one that helps the model solve the problem and helps the user understand the answer. The two are not the same and may be at odds.

The literature suggests that some of the most performance-improving reasoning is also some of the least faithful. Models that produce long, polished CoT explanations may be using the CoT instrumentally (to allocate compute, to commit to a path) while the underlying causal structure of the answer is different. Exposing a summary to the user gives them something to read; treating it as an explanation gives them something to be misled by.

The cleanest design position is the one Anthropic and Google have effectively converged on for newer models: expose a summary as a UX affordance, expose a signature as a correctness artifact, and document the gap. The least defensible position is "this is what the model thought" without qualification.

Billing legibility versus computational reality

Computationally, reasoning tokens are real generated work. Billing them as output tokens is economically coherent — the vendor pays for the inference compute regardless of whether the client sees the text. But billing hidden work makes cost estimation harder for clients in two ways: the work is not bounded by what the client requested in any tight sense, and the work is stochastic enough that even repeated calls on identical inputs can vary by a factor of 2–5x in reasoning tokens.

The alternative — flat-rate pricing per reasoning task — has not been adopted by any major vendor, presumably because it would force the vendor to absorb the variance and would create incentives to game effort settings. The current design pushes variance onto the client, which is correct on incentives and painful on budgeting.

Replayability versus statelessness

Stateless APIs are easier to scale, easier to reason about for security, and easier to debug. But reasoning continuity is fundamentally stateful: the model's reasoning at turn N depends on its reasoning at turn N-1 in a way that does not reduce to message history.

Vendors have split the difference by inventing portable state artifacts: OpenAI's encrypted_content, Anthropic's signed thinking blocks, Google's thoughtSignature. These artifacts let the client carry server-side state across calls without the server retaining it. The trade-off is that the client now has to manage opaque state correctly — and the failure mode of "dropped the signature" is silent on Anthropic, a 400 on Gemini 3 function calling, and a quality degradation on OpenAI.

ZDR and equivalent compliance regimes amplify this. Zero-data-retention deployments cannot rely on previous_response_id; they must use encrypted_content exclusively. The orchestrator code paths for compliance-sensitive workloads are not the same as for ordinary ones, and treating them as the same will eventually produce a continuity failure.

Cacheability versus correctness

Prompt caching dramatically reduces cost for repeated-prefix workloads. Reasoning configuration is part of the cache key. The conflict is that aggressive cost optimization wants stable thinking configurations; quality optimization wants per-task tuning of reasoning effort. The two are not naturally aligned.

The most operational answer is to cluster workloads by task class, fix the reasoning configuration within a class, and accept that cross-class cache hits will not happen. This is what production orchestrators usually converge to, but it is not a free choice — it constrains how task routing can be designed. See Reasoning-Effort Routing for the corresponding orchestration pattern.

Practitioner rules

The operational conclusions, distilled to rules an orchestrator should encode:

Treat reasoning spend as a distribution, not a constant. Log per-task-class P50/P95/P99 of reasoning_tokens (OpenAI), billed output minus visible text (Anthropic), thoughtsTokenCount (Google). Budget against the percentile that matches the operation's cost-overrun tolerance.
Read usage fields, not response bodies, for cost. Counting tokens in the visible response systematically undercounts billed work, sometimes by an order of magnitude.
Reserve max_output_tokens headroom for hidden reasoning. A floor of about 25,000 tokens is a reasonable starting point for medium/high reasoning effort on hard tasks; tune down with telemetry.
Preserve provider-specific continuity artifacts verbatim across tool loops and retries. OpenAI reasoning items or encrypted_content; Anthropic thinking blocks (including redacted_thinking) with signatures intact; Google thoughtSignature blobs on the right content parts.
Fix reasoning configuration within a cache cohort. Treat reasoning.effort, thinking.budget_tokens, and thinkingLevel as part of the cache key, not as runtime-tunable per call.
Distinguish four call outcomes in telemetry: success-with-visible-output, retry, hard-error, and reasoning-truncated (incomplete due to budget exhaustion). The last is silent if not explicitly tracked.
Treat thinking content and continuity artifacts as sensitive. Redact from default logs; apply retention policies; do not surface raw thinking to end users without product-level review.
Pick the lowest reasoning setting whose success rate is statistically indistinguishable from the next setting up. The right control is the smallest one that meets the success bar at acceptable cost; the academic literature is clear that more reasoning is not monotonically better on every task class.

The minimal experiment for arriving at these settings on a new orchestrator is described in the Reasoning-Effort Sweep companion entry: 50–100 representative tasks per class, sweep low/medium/high (or equivalent) across providers, log usage and success, and pick the floor that meets the bar.

What this does not resolve

Three open questions remain, and a practitioner article should not pretend otherwise.

The first is whether reasoning tokens should be exposed in a more inspectable form across vendors. The trend has been the opposite — more encryption, more summarization, more signature-based continuity — driven by safety-monitoring concerns more than by client convenience. The CoT-monitoring literature suggests this trend is likely to continue. The intellectually honest position is that the practitioner desire for visibility is in tension with the research finding that unoptimized CoT is a more useful safety channel; the vendors have prioritized the latter, and the case for that prioritization is empirically defensible.

The second is whether reasoning-token cost will fall fast enough to make the budgeting machinery moot. Per-token output prices on reasoning models have dropped substantially since the o1 launch in 2024; if the curve continues, the operational concern shifts from cost to latency. Latency budgeting for reasoning is qualitatively different — hidden reasoning is the dominant contributor to time-to-first-visible-token — and is treated separately in Latency Budgets for Reasoning Models.

The third is whether a cross-vendor abstraction layer for reasoning state is feasible. The current artifacts are structurally different enough that any abstraction layer leaks. The pragmatic stance is to write provider-specific orchestration code for reasoning paths and resist the temptation to unify them; the abstraction cost has historically exceeded the maintenance cost.

Companion entries

Core theory: - Chain-of-Thought Prompting - Faithfulness of Chain-of-Thought - Test-Time Compute and Inference Scaling

Practice: - Prompt Caching Strategies - Reasoning-Effort Routing - Reasoning-Effort Sweep - Latency Budgets for Reasoning Models - Multi-Model Agent Orchestration Patterns - Context Window Economics - Cross-Vendor Model Evaluation

Counterarguments: - Chain-of-Thought Monitoring as a Safety Channel - Visibility and Optimizability in Model Outputs - Encrypted State as a Continuity Primitive