Get started
01 / 05The ledger6 rulings · retrieved 2026-09-15
Case study · the builders' stack

The first workload was our own.

Vitna is built inside Convexity, a market intelligence terminal whose assistant routes every call by intent across two vendors. This is the actual routing ledger from its git history, the decision pattern Vitna exists to automate. Reconstructed and retrieved 2026-09-15.

Every line traces to a dated commit or an in-code decision comment. Nothing is projected or invented.

8 daysModel release to production default
5 daysPrice cut to production primary
1Candidate refused on measured evidence
Decisions / Convexity assistant · model routingReconstructed from git history · retrieved 2026-09-15
6 rulingsTwo vendorsRouted by intentOldest first
  1. 01
    The assistant ships all-AnthropicHaiku 4.5 on classification and lookups, Sonnet 4.6 on chat and synthesis, Opus 4.6 on deep research, GPT-4.1 held as the cross-vendor fallback.
  2. 02
    The cheap tier crosses vendorsgpt-5.4-nano takes classification and lookups as the default; the incumbent becomes the fallback lane.
  3. 03
    Opus steps 4.6 to 4.7A coordinated default flip across every deep-research surface.
  4. 04
    Sonnet 5, eight days after releaseThe June 30 release clears evaluation and becomes the default for chat and synthesis; Opus moves to 4.8.
  5. 05
    Luna, five days after the price cutThe July 30 cut makes Luna the synthesis primary, reversible by flipping one environment variable. The internal rubric read Sonnet 8.0 to Luna 7.9, at roughly ten times the cost per token.
  6. 06
    The lineup settles; Terra is refusedOpus 4.8, Sonnet 5 and Luna stay. Nano and Haiku retire. Terra is declined on evidence: Sonnet's input price, 20 percent more on output, lower measured intelligence, and its one edge, speed, bottlenecks nothing here.
02 / 05The measurementsFour internal evals · tier read
The measurements

One day of measurement, and the ruling it produced.

Four internal evals, verbatim from the savings ledger, behind the entry that made Luna the synthesis primary. Scores are printed with their n and their date, and costs appear only as ratios, never as absolute spend.

Paired eval · 4 Aug 2026

96 generations: 12 tickers x 4 synthesis sections x 2 models, identical prompts

nanoluna
Compliance filter pass rate97.9%97.9%
7-dimension rubric, same outputs3.87.4
Cost per call, relative1.0x1.1x

The compliance gate passed an inverted P/E reading without complaint. Aggregate filters measure compliance, not correctness; the rubric is where the gap shows.

Reasoning-effort sweep · 4 Aug 2026

84 generations: 3 tickers x 4 sections x 6 reasoning tiers, identical prompts

low · coverage 8 · bull 7 · bear 8 · criterion 8 · neutrality 8 · 1.0x cost · 6.6s

Quality is not monotone in reasoning budget, and cost is: xhigh pays 3x the per-call cost of low to score lower, and loses framing neutrality on the way. low ties high at half the cost and 10 seconds less latency, and was adopted.

Tier read · 4 Aug 2026

6 generations: 1 thesis x 6 reasoning tiers, one shared bundle of 9,244 characters, byte-identical context

  • nano: 491 prose tokens, 0 reasoning tokens, 18.1s
  • none: 464 prose tokens, 0 reasoning tokens, 4.6s
  • low: 487 prose tokens, 105 reasoning tokens, 5.7s
  • medium: 450 prose tokens, 512 reasoning tokens, 9.6s
  • high: 487 prose tokens, 2,047 reasoning tokens, 20.9s
  • xhigh: 551 prose tokens, 4,095 reasoning tokens, 37.3s

Prose runs 450 to 551 tokens across every tier; reasoning runs 0 to 4,095.

Where each capability switches on · 4 Aug 2026

Same read, same six generations: what the output contains at each tier. The first tier to carry a capability is ruled under.

Capabilitynanononelowmediumhighxhigh
Circular-financing bear casenoyesyesyesyesyes
Supply-constraint evidencenoyesyesyesyesyes
Peer competitive marginsnonoyesyesyesyes
Non-circular decision criterionnopartialyesyesyesyes
Reports that an empty source carries nothingnononoyesyesyes

Prose is 450 to 550 tokens at every tier including nano, and reasoning spans zero to 4,095: nothing about effort makes the model write more, it makes it think more. low is where peer data and a falsifiable decision criterion appear; medium is where the model stops citing a source for claims it cannot carry. Above medium the returns collapse: high and xhigh spend 4x and 8x the reasoning tokens to add one mention and slightly fuller peer margins, at 2x and 4x the latency.

03 / 05Tail and rerun exhibitsExhibits 01 and 02
01 / 02One more measurement · 4 Aug 2026

The median lies about the tail.

  • Fastest at the median, and the cheapest
  • One call in twenty spikes to 8.8x its median
  • For anything interactive, the tail is the number
One scale · seconds per call · paired eval, n = 484.3s38.0smedian4.3sp907.2sworst call38.0sDrawn to scale: the worst call ran 8.8x its own medianThe tail is the number to watch
nano latency distribution4 Aug 2026
Samplenmedianp90worst
Paired eval484.3s7.2s38.0s
Effort-sweep control124.2s··
Tier read1··18.1s

nano at the median is the fastest configuration measured, and the cheapest. Its defect is tail variance: roughly one call in twenty spikes to 4-9x its median. For a four-hourly batch neither number matters much; for anything interactive, the tail is the number to watch.

Drawn to scale from the paired eval, n = 48: the worst call ran 8.8x its own median.

02 / 02The rerun · 4 Aug 2026

The prompt moved the curve. The budget moved the bill.

  • 15 of 15 gates, at every tier including zero reasoning
  • 0.5 between best and worst, from 1.2 under the old prompt
  • medium, high, xhigh no longer fit the 2,000-token cap
Two sweeps · overall score of 10 · axis 6 to 9Old promptRevised6789nonelowmediumhighxhigh6.67.87.27.86.87.57.98.07.87.91.2Old0.5RevisedDrawn to scale: 1.2 between best and worst became 0.5Correctness came from the instruction
revised instruction x tier15 arms: 3 tickers chosen for spread geometry x 5 reasoning tiers, the revised instruction, luna only, identical prompts4 Aug 2026
Tieroveralloutput tokenslatencyfits 2,000
none7.5426 to 4804.4 to 5.2syes
low · adopted7.9618 to 6656.9syes
medium8.02,020 to 2,09519 to 22sno
high7.83,510 to 4,62430 to 39sno
xhigh7.98,170 to 10,29666 to 81sno

none, at zero reasoning tokens, clears every gate: correctness came from the instruction, not the reasoning budget, and the old sweep was largely measuring which tiers rediscovered unstated requirements. The curve flattened to half a point between best and worst. The mandate roughly tripled reasoning tokens at a fixed tier, so medium now exceeds the production output cap in 3 of 3 tickers, and xhigh runs 66 to 81 seconds at about 8x the cost of low for at most a tenth of a point. Configuration settled: luna at low, with the revised instruction.

Compliance gates, every arm: 15 of 15. Scored by the author of the prompt against pre-registered gates, n = 3 tickers. Gate results are near-objective; graded means carry at least 0.3 of noise either way.

04 / 05After the last ruling4 readings · usage ledger
After the last ruling

The table stopped moving. The record did not.

The lineup settled on 10 Aug 2026 and the routing table in the repository has not changed since; the retrieval on 2026-09-15 confirmed it. What changed is what the usage ledger could say about it: 4 readings, each traced to a merged pull request, and none of them a ruling.

Readings / Convexity assistant · productionTraced to merged pull requests · retrieved 2026-09-15
4 readingsUsage ledgerNot rulingsOldest first
  1. 01
    Nine days with no synthesis, and nothing alertedNot a decision. The Anthropic account went unbillable on 26 Jul and the circuit breaker stopped attempting Sonnet. The batch path had no cross-vendor fallback, so the four synthesis sections did not refresh for nine days: every cycle logged its failed refreshes and zero succeeded, and nothing alerted. The fallback landed on 4 Aug, and the first clean cycle ran on the cheap tier.convexity PR #1618 · n withheld
  2. 02
    The usage ledger learns which lane spent the moneyA surface is where in the product a call came from; a lane is what routes it, and one surface spans several lanes. Read by surface, deep research for 1 to 23 August showed 56 percent of its calls on nano and 4 percent on Haiku, neither of which that lane routes to: its lookups ran on the cheap tier and the row could not say so. The column that says which lane spent the money did not exist, so cost per lane, and any answer to whether a lane should change model, had nothing to group by. Added nullable with no backfill: null means not recorded, and never means a lane.convexity PR #2260 · n withheld
  3. 03
    Two vendors lapse at once, and one of them silentlyAnthropic bills out as a 400, which the provider adapter had recognised since an earlier lapse, so every failure on those two days alerted. OpenAI bills out as a 429 carrying a quota code, indistinguishable by status from throttling, so it classified as rate limiting and never reached alerting at all. The batch extractors died behind the silent one for three days. Both error classes were already retryable and both fell across to the other vendor, which is what kept every interactive surface up; a test now pins that equivalence.convexity PR #2691 · n withheld
  4. 04
    The extractor spent its whole budget thinkingA 2,400-token output ceiling reaches a reasoning model as the budget for thinking and answer together. Measured over 60 calls on 2 Sep, the first time the extractor had run outside a test: at that ceiling the model spent the budget thinking and returned an empty body on 35 of 40 calls, and the success path cached each empty result as a company that disclosed no guidance. Sixteen of the same documents re-run at a non-binding ceiling produced a body every time. Real demand measured 1,493 tokens at the minimum, 3,455 at the median, 4,283 at the p90 and 5,041 at the maximum. The ceiling rose to 8,000, an empty or truncated body now takes the failure path and stays retriable, and the surface gained a daily cap of its own.convexity PR #2654 · n = 60

Where a reading carries no n, the count is a call volume on one surface. Beside the token profile the register already carries for that lane and a public price, a volume is a spend figure, and this study keeps spend to ratios.

Never use a compliance filter pass rate as a quality signal.It rewards hedging and penalises directional analysis. Sonnet scored worst on it and was the best writer; the inverted valuation reading passed it.
Record the quality consequence with every saving.An 88 percent cost reduction that ships an inverted valuation reading to users is not a win.
Measure token composition, not just totals.Reasoning and prose tokens bill identically and mean opposite things.
A token ceiling is a ceiling, not a reservation.Raising it costs nothing; setting it low silently truncates.

4 standing rules the savings ledger wrote down for itself on the day of the measurements above, in its own words.

05 / 05Put your workload on recordDesign partner program

Put your own workload on the record.

The pattern above, run by hand for five months, is the one the engine automates: a threshold somebody armed, a run, a brief, and a ruling a person signs.

Design partner program

Bring one recurring workload.

Extraction, support, research synthesis: whatever recurs and can be measured. We map the decision, define the thresholds, and produce the first brief with you.

Request a workload audit