Before · claude-opus-4-8
$0.5112
per call, across 5,780 calls over 34 active days
After · claude-sonnet-5
$0.1528
per call, across 3,282 calls over 56 active days
Difference
70%
lower cost per call, sustained — not a launch-week figure
Windows are each model’s entire life on this workload, with the single day both served traffic excluded from both sides — no hand-picked bounds. Counts taken 2026-08-27 from production, and re-derivable: pnpm tsx scripts/capture-finautor-case-study.ts reproduces every number here or prints the difference.
What the telemetry shows about the switch itself
- One day where both models served traffic. On 2026-07-01, 39 calls went to claude-opus-4-8 and 128 to claude-sonnet-5. A deliberate cutover leaves that mark; an outage does not.
- And then none, ever. Zero claude-opus-4-8 calls after that day. The old tier was not kept as a fallback, which is what makes the after-window a clean measurement rather than a blend.
- 56 days, no gaps. The new tier carried traffic on every one of the 56 calendar days it spans. This is a habit, not a trial.
The migration used the method Bayeto now automates — find the traffic paying a premium tier for work a cheaper one handles, price the swap against your own rates, and measure it afterwards. Bayeto did not emit this recommendation at the time; the product was not yet named. What it can do is be run against the pre-switch telemetry today, which is the next section.
Did the answers get worse? Finautor's engineering team set the bar before the switch, then measured it
The gates below were declared on 2026-07-01 — before the flip, not after the result was known — with a named decision date and a revert path of a single environment variable, never used. Over 57 days and 695 primary turns the verdict was KEEP — every gate passed.
| Gate | Opus baseline | Sonnet, 57 days |
|---|---|---|
| Clean finish (end_turn) | 96.2% | 96.4% |
| Truncation (max_tokens) | under 2% | 0.14% |
| Synthesis failures | 1.09% | 0.58% |
| Refusals | 0 | 0 |
Answers also got shorter — 3,499 to 2,777 average output tokens, 21% less. That was not one of the gates, and it is part of why cost per call fell: a cheaper model that also answers more briefly is two effects, not one. The gates above are the check on whether the shorter answers still finished the job.
Whose numbers these are. Every cost figure on this page is re-derived from telemetry Bayeto holds. These are not: they were measured in Finautor's own production database, which Bayeto has no access to. Their own experiment also reports a cost saving of about 40% — a smaller number than the headline because it covers chat turns only, at list price, excluding cached and background calls. Both are right; they count different things, and neither is the other with a rounding error.
Run on the data as it stood before the switch, the engine proposes the same move
Bayeto re-runs its whole pipeline with the post-switch rows withheld and grades its own proposal against what actually happened. On this workload it proposes claude-opus → claude-sonnet — and it got the size wrong in the direction worth publishing:
Bayeto over-predicted this one. It said €0.1883 a call; the measurement said €0.1377 — about 5 cents a call too generous, 63% accurate. And the traffic that moved was a fraction of what it had been, so the comparison is drawn over a much smaller population than the prediction assumed.
That figure grades the engine’s estimate against a counterfactual repricing of the traffic it named. It is a different comparison from the headline above, which is raw cost per call across the whole workload — the two are not meant to reconcile, and we would rather say so than have you try.
The second move: what a randomised control did to our prediction
Bayeto predicted that pruning 15% of a compressible prompt prefix would cut this application’s prompts. Finautor's engineering team ran a randomised concurrent control — 37 calls treated against 36 untreated, split 49%/51% over 130.8 hours — and measured it.
Prompt tokens — a real effect
−12.2%
95% interval −20.7% to −2.8% — it excludes zero, so the effect is there.
Cost per request — not yet resolvable
−6.9%
95% interval −26.0% to +17.1% — it contains zero. Another week of data.
We predicted 15%; a control measured −12.2%. The cost effect is not yet resolvable and we are not going to call it one. That distinction is the product. An engine that cannot tell a result from a hope will happily bill you for the hope.
What this does not claim
- Not that Bayeto recommended the migration at the time — it did not exist. It proposes the move when run on the pre-switch data, which is a different and checkable claim.
- Not that Bayeto measured quality — it does not, and the method page says exactly which checks run and which do not. Finautor measured it themselves, against gates they set before the switch; that section is above, and those figures come from their system rather than ours.
- Not an independent customer. Finautor is the founder’s other company, which is why its data could be used and named here at all. The telemetry is real production traffic and the figures are re-derivable; the relationship is disclosed so you can weigh it.
Find the same thing in your own telemetry
One usage export, parsed in your browser. 9,062 calls is all it took to measure this one — or see what Bayeto looks for first.