The Reasoning Floor: Methodology, Cell Data and Control Probe

Design

The question was whether the output token volume of one benchmark arm belonged to the model or to a serving accident, and whether any client parameter could reduce it.

Substrate held byte identical to the original run: the same production move prompt from a browser word game, the same parser and validator, the same twelve deterministic seeded positions, temperature 0.4. The prompt, parser, validator and game engine were confirmed unchanged in version control since two days before the original benchmark, so nothing in the grader moved underneath the comparison.

Every arm is hard pinned with an explicit single provider and fallbacks disabled, and the serving provider is recorded per cell, so the artifact proves the route rather than assuming it. All 48 cells completed with zero transport errors.

† Survivorship. B2's means describe only the four calls that completed and must not be quoted as that endpoint's behaviour.

The control validates the instrument

Before reading anything into the reproduction: the reference arm moved from 8 of 12 to 7 of 12 legal and from 1,125 to 1,190 mean output tokens between the two runs, which is inside the original pre-registered tie band and within 6% on tokens. The harness did not drift, so the comparison is between models rather than between instruments.

Token volume reproduced, failures did not

Cell for cell on the same route, fourteen hours apart:

Individual cells swing hard in both directions, which is expected of a single trial instrument at temperature 0.4 with a documented two in twelve replication spread. The aggregate is nearly exact: 28,144 to 28,031 mean output tokens, $0.007896 to $0.007865 per call, 99.8% reasoning share in both.

All five failures came back legal, including both cells that had hit the 64,000 token ceiling. A five in twelve legality swing exceeds the documented noise band, but it is still one trial per cell, so the defensible statement is narrower than an improvement factor: at the same spend per call, the route now returns a usable answer where it previously often returned nothing.

The mechanism

The model card for this build states that the release does not include a chat template. Instead the vendor ships an encoder implementation and asks each serving provider to implement the prompt format itself. The reasoning control is not a sampler setting: it is a text prefix.

There is no medium. A separate mode closes the thinking block immediately and suppresses the reasoning channel entirely.

With no canonical template, how much reasoning a caller is billed for is a decision each provider makes on their behalf, and providers can disagree by an order of magnitude on byte identical requests. That is not a defect in any one provider. It is the predictable consequence of shipping a reasoning model without a chat template, and it also explains why several third party endpoints for this build sat deranked in the broker's catalog while their operators worked the serving out.

The control probe

Prompt held fixed, reasoning control varied, two mid weight positions, $0.03499 of spend.

Two findings, closing the question from opposite directions.

Asking for low effort does nothing, because low is the documented default and prepends no prefix, so requesting it is a no-op. The numbers agree: a 6% move on one position, and on the other the three reasoning enabled configurations land at 7,305, 10,823 and 13,827 with no monotone ordering, a spread of roughly 30% that reads as run to run variation.

Turning reasoning off relocates the thinking rather than removing it. The channel goes cleanly to zero and both cells become illegal: one returns a 253 token stub that fails to parse, and the other runs its full 32,000 token budget into the ceiling with no reasoning channel at all, having inlined its chain of thought into the message body without ever reaching the required JSON.

One transport error occurred on the nested parameter cell of p08 and was not retried, because the flat parameter cell on the same position answers the same question.

What this did not settle

1. Build versus serving. The second genuine endpoint was included precisely to separate these, and it hard failed 8 of 12 calls, returning an error finish reason with zero prompt tokens and zero cost after ten to twenty minutes each, having generated fifteen to twenty eight thousand reasoning tokens first. So the claim rests on one provider measured twice, not on two providers agreeing. What would settle it: rerun against the other endpoints once they leave deranked status, or self host briefly against the published weights with the reference encoder.
2. You cannot conclude the failing provider reasons less. Its lower mean covers only the four calls that completed, which is exactly the subset that did not run away.
3. Why the failures cleared is unexplained. Candidates include ordinary variance, the vendor stabilising a day old endpoint, and this run using lower concurrency. Not separated.
4. Single trial per cell throughout, at temperature 0.4, on an instrument with a documented two in twelve replication spread.
5. Task family. Constrained JSON emission for one game move is close to the worst showcase for a build whose headline improvement is agentic work. A large deliberation budget is a liability here and may be an asset there. That was not measured.

Spend

$0.242457 measured from per call provider reports: a build identity probe at $0.000277, a smoke test at $0.004470, four main grid arms totalling $0.202720, and the control probe at $0.034990.

The broker's account ledger moved $0.4738 across the same session, a gap of about $0.231 that could not be attributed. Candidates: an always on speech service drawing on the same key, the eight failed calls that reported zero cost to the client but may have billed upstream, and an abandoned first control probe that was killed after roughly thirty minutes without completing a call. Anyone costing this build from these figures should use the ledger delta rather than the per call sum.