An OpenAI-compatible router you run yourself. Send model: "auto" and written rules choose the model and the reasoning effort for each request without asking a model, keep a warm conversation on its cache, and write down why.
In shadowIt runs on one server, for Convexity, and nothing on this page installs yet.
A requestin amber
Our payments service double-charges about one customer in 20,000. Walk through, step by step, every way an at-least-once queue, a retrying HTTP client and a non-idempotent charge endpoint can combine to cause it; prove which one fits a 1-in-20,000 rate given these numbers: 0.3% client retry rate, 0.02% queue redelivery, 40 ms p99 timeout; and design a fix with an idempotency key, including the exact schema, the race you must still handle, and a test that proves it.code, difficulty 43, precision 2
strongdifficulty 70 to 100
anthropic/claude-opus-5
None of the four goes this far.
standarddifficulty 40 to 69
anthropic/claude-sonnet-5
A payments bugstandard · mediumdifficulty 43 · ~$0.02124 a turn
A migration designstandard · lowdifficulty 40 · ~$0.01511 a turn
lightdifficulty 0 to 39
deepseek/deepseek-v4.1-flash
A quick questionlight · highdifficulty 16 · ~$0.00402 a turn
A thank-youlight · lowdifficulty 0 · ~$0.00055 a turn
Four requests written for this page, each decided by the router in this repository when the site was built. The picture is drawn, not measured: requests pass the router's ladder and leave on its three tracks. In amber, the request taken apart below.
+1.03 msadded at the median, one 1 KB chat request at a time, against an upstream that answers instantly
17 of 20held-out requests routed to an accepted tier, labelled before the first run and never tuned against
237tests, including a real router on a port in front of a scripted upstream
8 of 8deliberate breaks of its core properties turned the suite red
02 / 08How it decidesSix rules · no model call
How it decides
One decision, taken apart.
Six steps run between a request and its model, and none of them asks a model. The decision is a function of the request and the conversation it continues, so it can be run again and give the same answer, which is how this panel was made.
Our payments service double-charges about one customer in 20,000. Walk through, step by step, every way an at-least-once queue, a retrying HTTP client and a non-idempotent charge endpoint can combine to cause it; prove which one fits a 1-in-20,000 rate given these numbers: 0.3% client retry rate, 0.02% queue redelivery, 40 ms p99 timeout; and design a fix with an idempotency key, including the exact schema, the race you must still handle, and a test that proves it.
Class
code
Difficulty
43of 100
Precision
2of 3
What it was scored on
class:code+28
reasoning_cues+5
analysis_cues+3
rigor_words+7
precision: precision_words+1
precision: exact_classes+1
Decision
anthropic/claude-sonnet-5
effort mediumtier standard
difficulty 43 -> standard/low
precision 2 raises effort to medium
no warm conversation
If a call fails, in this order
anthropic/claude-sonnet-5medium
openai/gpt-6-solmedium
anthropic/claude-opus-5medium
~2,100 output tokens, ~$0.02124 this turn, with no cache warm
The request
We run Postgres 14 behind PgBouncer. Design a zero-downtime migration that splits a 2 TB orders table by region: the cutover plan, how to backfill, how to verify row counts, and how to roll back at each step.
Class
analysis
Difficulty
40of 100
Precision
0of 3
What it was scored on
class:analysis+30
analysis_cues+3
rigor_words+7
Decision
anthropic/claude-sonnet-5
effort lowtier standard
difficulty 40 -> standard/low
no warm conversation
If a call fails, in this order
anthropic/claude-sonnet-5low
openai/gpt-6-sollow
anthropic/claude-opus-5low
~1,500 output tokens, ~$0.01511 this turn, with no cache warm
The request
What does HTTP status 429 mean?
Class
qa
Difficulty
16of 100
Precision
0of 3
What it was scored on
class:qa+16
Decision
deepseek/deepseek-v4.1-flash
effort hightier light
difficulty 16 -> light/high
no warm conversation
If a call fails, in this order
deepseek/deepseek-v4.1-flashhigh
openai/gpt-6-lunahigh
openai/gpt-5.6-lunahigh
~3,350 output tokens, ~$0.00402 this turn, with no cache warm
The request
Thanks, that worked!
Class
chitchat
Difficulty
0of 100
Precision
0of 3
What it was scored on
class:chitchat0
Decision
deepseek/deepseek-v4.1-flash
effort lowtier light
difficulty 0 -> light/none
deepseek/deepseek-v4.1-flash takes low, not none
no warm conversation
If a call fails, in this order
deepseek/deepseek-v4.1-flashlow
openai/gpt-6-lunalow
openai/gpt-5.6-lunalow
~460 output tokens, ~$0.00055 this turn, with no cache warm
Four requests written for this page, each decided when the site is built by the router in this repository with the config it ships, making the calls switchyard explain makes: as the first turn of a conversation, so no cache is warm, and with no preference from the caller. The command prints these fields as text, and the layout is ours. Costs are at the config's prices, read 2026-09-28, and deepseek/deepseek-v4.1-flash's on 2026-09-30.
1FeaturesReads the request once and returns counts: message sizes, code, stack traces, math notation, cue words by family, constraints, media, tools, conversation depth. No text survives it.src/features.mjs
2ScoresA task class, a difficulty from 0 to 100 and a precision from 0 to 3, under weights written in the config. A short follow-up is scored by the task it continues.src/score.mjs
3LadderThe difficulty picks a rung, and the rung a tier and an effort. A caller's preference slides the request along the line, and precision raises the effort's floor.policy.ladder
4CapabilitiesThe first model on the track that can serve the request: images, audio or files, tools, structured output, the context window, the output cap.per model, in the config
5StickinessA warm conversation keeps its model and its effort, so its cache stays warm, unless moving is worth the cold start.src/policy.mjs
6FallbacksThe chosen model, its tier-mates, then one tier up, never a cheaper tier. A malformed request and a refusal are not retried elsewhere.429 · 5xx · timeouts
The ladder and the tracks, as shippedread from the config this repository ships, the one the benchmark measured at commit e27ca78d
The ladderpolicy.ladder
as shipped
Difficulty
Tier
Effort
0 to 9
light
none
10 to 24
light
high
25 to 39
light
high
40 to 54
standard
low
55 to 69
standard
medium
70 to 84
strong
medium
85 to 100
strong
high
A caller's preference moves a request along the line: cost -10, balanced 0, quality +10. Precision raises the effort's floor, as it did for the payments bug above.
The trackstiers
up on a failure, never down
light
deepseek/deepseek-v4.1-flash
openai/gpt-6-luna
openai/gpt-5.6-luna
anthropic/claude-haiku-4.5
standard
anthropic/claude-sonnet-5
openai/gpt-6-sol
strong
anthropic/claude-opus-5
openai/gpt-6-astra
The ladder and the tracks are read from the config this repository ships, switchyard/config/router.config.json, and are the ones the benchmark measured at commit e27ca78d.
03 / 08The benchmarkMT-Bench · MT-Bench-101
The benchmark · MT-Bench, then MT-Bench-101
Against Jev Router, judge by judge.
MT-Bench's public questions, answered through OpenRouter from 2026-09-28 to 2026-09-30 and graded blind by three judges from three vendors, then 80 questions from MT-Bench-101 that nobody here had read, to see whether the result held. Both are chat, not anyone's traffic.
80questions per turn in each set, first turns and follow-ups
2,547answers, graded blind 2,208 times
3judges decide, Grok the neutral one; three more graded beside them, none in the mean
$39.01spent on everything in the record
Exhibit 02 · Against Jev Router
Switchyard since 2026-09-30 DeepSeek V4.1 Flash leads its light tierheld up, judge by judge · 2026-09-28 to 2026-09-30
Where Jev Router leads comes first. It answers faster, it costs less on MT-Bench-101's short questions, and the GPT judge puts it ahead on follow-ups: the lit cells in its columns. Everywhere else in this table Switchyard is ahead.
The table scrolls sideways.
Held up, by
MT-Bench, first turnsof 80
MT-Bench, follow-upsof 79
MT-Bench-101, first turnsof 79
MT-Bench-101, follow-upsof 75
Switchyard
Jev Router
Switchyard
Jev Router
Switchyard
Jev Router
Switchyard
Jev Router
Mean of the three judgeswhat the rules were decided on
66
54
68
62
70
34
73
50
Claude judge
70
56
69
54
62
26
69
43
GPT judge
69
64
54
58
72
70
67
68
Grokneutral
60
41
66
60
65
27
68
38
Qwenneutral, not in the mean
53
35
54
45
53
16
58
26
DeepSeek V4 Proan interested judge, MT-Bench only
50
45
52
44
not asked
not asked
Gemini 3.8 Flashan interested judge, MT-Bench only
64
50
62
58
not asked
not asked
Cost of the answersthe total for each turn's answers
$0.117
$0.123
$0.215
$0.288
$0.056
$0.028
$0.052
$0.033
Median time to answerDeepSeek's answers, for Switchyard
6.4 s
3.6 s
11.0 s
5.1 s
not recorded
not recorded
The brighter figure of each pair is ahead: more held up, less cost, less time. An answer holds up when it scores within one point, on a scale of 1 to 10, of the best answer the tiers gave to the same prompt. Out of 80 and 79 on MT-Bench and 79 and 75 on MT-Bench-101, except where a judge graded fewer on MT-Bench: the Claude judge and Qwen graded 77 and 73 of the follow-ups, DeepSeek V4 Pro 77 first turns and 56 follow-ups, and Gemini 3.8 Flash 79 and 78.
Which judge you ask moves the frontier. Five policies on MT-Bench under each reading. On the six-answer run the Claude judge put Opus 22 first-turn answers ahead of gpt-6-luna at high effort, and the GPT judge put it 14 behind, so a claim across vendors needs every column. It opens on the neutral judge's follow-ups.
Turn
Held up by
Switchyard's rules, since and before 2026-09-30
Jev Router, and one model always: DeepSeek V4.1 Flash, or Opus 5
The frontier: nothing holds up more for less
Follow-ups, held up by Grok, the neutral judge
Not drawn: the best answer each time, 79 of 79 for $0.091, chosen in hindsight.
Follow-ups, held up by the Claude judge
Not drawn: the best answer each time, 79 of 79 for $0.091, chosen in hindsight.
Follow-ups, held up by the GPT judge
Not drawn: the best answer each time, 79 of 79 for $0.091, chosen in hindsight.
Follow-ups, held up by the mean of the three judges
Not drawn: the best answer each time, 79 of 79 for $0.091, chosen in hindsight.
First turns, held up by Grok, the neutral judge
Not drawn: the best answer each time, 80 of 80 for $0.047, chosen in hindsight.
First turns, held up by the Claude judge
Not drawn: the best answer each time, 80 of 80 for $0.047, chosen in hindsight.
First turns, held up by the GPT judge
Not drawn: the best answer each time, 80 of 80 for $0.047, chosen in hindsight.
First turns, held up by the mean of the three judges
Not drawn: the best answer each time, 80 of 80 for $0.047, chosen in hindsight.
where Jev Router leadsFaster, cheaper on short questions, and ahead under the GPT judge on follow-ups.A median 3.6 s and 5.1 s on MT-Bench against DeepSeek's 6.4 s and 11.0 s. On MT-Bench-101 it cost $0.028 and $0.033 against $0.056 and $0.052, sending most questions to gpt-6-luna. The GPT judge: 58 against 54 and 68 against 67 follow-ups.
MT-BenchMore held up than Jev Router on both turns, for less.66 against 54 and 68 against 62 by the mean, and ahead under Grok and the Claude judge too, for $0.117 against $0.123 and $0.215 against $0.288. That is 2.3 and 2.4 times what the rules it replaced cost.
fresh questionsAnd on questions nobody here had read, by more.70 against 34 and 73 against 50 by the mean, and ahead under Grok and Qwen. DeepSeek was chosen from MT-Bench's own prompts, so 10 dialogues from each of eight MT-Bench-101 tasks were drawn before any answer existed, and all five predictions written down with them held.
the GPT judgeOne judge disagrees.The GPT judge prefers the rules the router shipped before 2026-09-30, when gpt-6-luna led the light tier, on both sets and both turns, and it is the only judge that rates luna's answers near the top. Every other judge on this page puts the rules since ahead on both turns of every set it graded, the two neutral ones among them.
the router itselfOne model, always, did as well, for less.On MT-Bench always DeepSeek held up on 68 and 70 against 66 and 68, within the judges' noise, for $0.098 and $0.168: the rules' escalations to Sonnet 5, 7 first turns and 6 follow-ups, cost money and did not help. On MT-Bench-101 the rules sent every question to DeepSeek, so there the two are the same policy.
These five are the whole of what the record says its tables support. Read them with the terms above and with these: both sets are chat, not your traffic; eighty questions per turn, so a gap of two or three answers is within noise; the MT-Bench follow-ups are out of 79; the judges prefer their own vendor's answers; and Vitna built Switchyard and ran the benchmark. Every table, the predictions and all ten caveats are in the full record.
04 / 08Built to be checkedShadow · report · replay · route
Built to be checked
It shows its work.
A router you cannot read is a router you cannot trust with your traffic. This one writes down every decision, starts by watching rather than serving, and never trades a failure for a cheaper answer.
Shadow first, then route
the default
The incumbent keeps serving while the rules' pick is written down. Read what they would have done, try a new policy on the traffic already recorded, and only then does a person throw the switch.
ShadowSWITCHYARD_MODE=shadow
Reportswitchyard report
Replayswitchyard replay --config candidate.json
RouteSWITCHYARD_MODE=route
It writes down why
counts and hashes
One row per request, written from a finally so a request that fails still leaves it: the features as counts, the scores and the decision with their reasons, every attempt, the tokens and the cost, and where the milliseconds went.
It never keeps the prompt, the completion or an error message, so a row can be read, shared and replayed without exposing a word.
A warm conversation keeps its cache
cache kept
A provider drops a conversation's cached prefix when the model or the reasoning effort changes, so Switchyard keeps both on a warm conversation unless moving is worth the cold start. It needs no session id: a turn continues an earlier one when the earlier turn's messages are a prefix of its own.
A failure moves up, never down
fallbacks
A failed call moves along the same track, then one tier up, and never to a cheaper tier: a failure is not a reason to answer a request worse. A malformed request and a refusal are not retried elsewhere.
05 / 08The othersMeasured vs documented
The others
Only what was measured gets a score.
Jev Router is the one other router Switchyard was run against. OpenRouter's Auto Router and Not Diamond also choose a model per request, and are here as their own documentation describes them, read on 2026-09-29, with no figure, because nothing here asked either of them anything.
Four routers, one measured against anotherread 2026-09-29
no scores for what was read
The table scrolls sideways.
Property
Switchyardmeasured here
Jev Routermeasured here
Auto Routerread, not run
Not Diamondread, not run
What it decides
The model and the reasoning effort for each request sent as auto
picks the best model and reasoning effort for each request
The model for each request, within a cost tier the caller sets
The model for each request
How it decides
Rules over counted features of the request, no model call
runs on Jev, TypeSafe's first System One model
A classifier assigns each prompt one of about thirty task types, then ranks models by what OpenRouter's users spent on for that type over a trailing seven days
A router trained on the customer's own evaluation data: prompts, each model's response and a score for each, at least 25 samples
What a decision costs
None; about a millisecond in process
The listing prints no price of its own
There is no additional fee for using the Auto Router
Not in the document read
What it says about a choice
Every decision carries its reasons
Not in the document read
The response's model field names the model that answered
Can I run it?Not yet. It runs in shadow on one server, and nothing on this page installs.
What runs todaynothing on this page installs
In shadow, on one serverin serviceIt answers on loopback only, for Convexity, and serves every auto request with the incumbent while it records its own decision.
The rules since 2026-09-30in this repositoryDeepSeek V4.1 Flash leads the light tier, the rules the benchmark measured. The server in shadow changes its decisions only when a release is published to it, and which release it runs is read on that server, not from here.
DeepSeek in route modenot yetBefore it serves a request, DeepSeek has to be pinned to providers that neither retain nor train on prompts, and the router has no per-model provider setting yet. Whether it takes tools while reasoning, through OpenRouter, has not been checked.
Installablenot yetIts repository is private and there is no package or image.
The namenot settledA working name. No trademark search has been run, and Red Hat's JBoss project SwitchYard exists.
Beside the runtimenot part of itvitna-runtime keeps its own per-request rules, declared on the request's shape and signed into a policy. Switchyard reads each request's text in process to count it, and keeps only the counts.
When does a warm conversation move?the stickiness rules, as the router's README states them
When a warm conversation movesfrom the README
cache kept
When
What it does
The task needs a bigger tier
upgrademove, forfeiting the cache
Same tier, effort two or more steps higher
upgrade_effortsame model, new effort
Same tier otherwise
stickysame model and effort
A cheaper tier would do, but the task continues
sticky
The task changed and moving saves at least downgradeMargin over horizonTurns, cold start included
downgrade
The task changed but the warm cache makes staying cheaper
sticky_cost
Past the TTL, or the model cannot serve the turn, or the caller excludes it
cold, ineligible, constrainedfresh pick
upgradeEffortSteps 2
downgradeMargin 0.2
horizonTurns 3
The rows are the README's, copied at the pinned commit. A result in periwinkle keeps the cache and one in amber pays for a cold one. That a change of model or effort drops the cache is the providers' own rule: Anthropic's and OpenAI's prompt caching documentation, as Switchyard's decision log read them on 2026-09-28.
What does it keep about a request, and what never?counts and hashes, never the words
What a row holds
counts and hashes
featuresas counts
scoresand their reasons
decisionand its reasons
chain hashesHMACs, under the deployment's own key
attemptsevery upstream try, with its status
usagethe upstream's tokens and cost
timingswhere the milliseconds went
Jevits answer, as numbers, when asked
What it never holds
never
prompt textFeatures are counted, and the words are not kept.
completion textAn answer goes to the caller and nowhere else.
error messageA moderation error can quote the prompt, so only a type label is kept.
Two of its test files plant a word in a request and assert it never reaches the ledger. A change to how features are counted bumps a version, and replay skips the rows it can no longer compare.
What does the router add to a request?+1.03 ms at the median, one request at a time
Every scenario measured2026-09-28
c is requests in flight
Scenario
Added p50
Router p99
Throughput
Chat, named model, 1 KB, c=1
+1.03 ms
2.83 ms
732 req/s
Chat, named model, 1 KB, c=16
+8.35 ms
22.9 ms
1,532 req/s
Chat, named model, 1 KB, c=64
+30.8 ms
63.3 ms
1,770 req/s
Chat, auto in shadow, 1 KB, c=16
+7.20 ms
15.2 ms
1,885 req/s
Chat stream, 1 KB, c=16
+7.83 ms
18.0 ms
1,640 req/s
Chat, 200 KB, c=1
+25.0 ms
38.3 ms
38 req/s
Chat, 200 KB, c=4
+99.6 ms
133 ms
39 req/s
Anthropic messages, 1 KB, c=16
+6.51 ms
14.3 ms
2,060 req/s
Jev on, chat 1 KB, c=1
+1.20 ms
4.28 ms
686 req/s
Jev on, chat 1 KB, c=16
+11.0 ms
29.8 ms
1,222 req/s
Jev on, Anthropic messages, c=16
+9.31 ms
20.9 ms
1,453 req/s
Measured against an upstream that answers instantly, each in a process of its own, so every millisecond of difference is the router's; with more in flight it is mostly queueing on one Node thread. The full run of 2026-09-28, on a Windows 11 ARM64 laptop, Node 24.14, not the production server, with other work sharing it: both sides of every comparison ran under the same load, so the differences hold better than the absolute numbers. The router's p99 at one request at a time was 2.83 ms. It predates a faster pass over the features, which took a 200 KB prompt from +37.5 and +38.0 ms to +15.8 and +16.0 ms one at a time.
What is Jev, and does it route anything?a second opinion, asked beside the rules, that routes nothing
Jev is TypeSafe's decision model. With Jev on, Switchyard asks it one question per request, how capable a model the request needs, alongside the model call, so a caller waits for nothing and a slow or failing Jev changes nothing. The ledger keeps its answer as numbers.
What asking costs
sends text to TypeSafe
Calls
59
Cost, all of them
$0.002
Median
172 ms
p95
372 ms
It is off in the config this repository ships. Where it is on, the text a request is scored on also goes to TypeSafe, through OpenRouter; a client listed in jev.optInClients is asked about only the requests it marks, and Convexity's backend marks only its public-data surfaces.
It leans high: where it and the rules disagreed, 11 times, it chose the bigger tier 9 times.
Routed to an accepted tier2026-09-28
a smoke test
The labelled requests, measured on typesafe/jev-1.13-20260917 with the question as written and never tuned. A smoke test, not a benchmark.
Held out (20)
the rules17 of 20
Jev19 of 20
the higher of the two19 of 20
In-sample (39)
the rules39 of 39
Jev38 of 39
the higher of the two38 of 39
Whether it has earned a vote is in what the benchmark taught, in the full record. So far it has not.
Can it speculate on keystrokes, the way a 0.5-second agent does?0.49 ms median across 869 sections, in the browser, without sending unsubmitted words anywhere
Sanity's engineering write-up ("The 0.5-second agent", sanity.io/engineering/the-0-5-second-agent) shows that an agent feels instant when entry selection, source choice, model routing, and document staging finish during typing pauses (250 ms debounce) before the reader presses Enter. Sanity asks TypeSafe's Jev over the network on every pause (282 ms for one question, 340 ms for three bundled questions, $0.00008 per pause). Because Switchyard's scorer and policy engine are pure functions over an inverted token posting index with zero dependencies, the documentation shell runs all three decisions in the browser in 0.49 ms at the median (1.23 ms p95 across 869 sections), for $0, while keeping unsubmitted keystrokes on the reader's machine.
Open /docs and choose Ask agent (0.5s) in the rail. Every 180 ms typing pause scores candidate postings (staging two or three pages when probability splits across concepts or multi-hop comparisons, bridging HTTP status symptoms like 429 and 503, or withholding when none_of_these leads), picks the source surface, and runs src/policy.mjs in process (0.24 to 2.29 ms warm on POST /docs/agent.json).
Sanity's 0.5s Jev stackpublished figures
network per pause
3-question Jev call
340 ms
Cost per pause
$0.00008
Haiku tool baseline
1,328 ms
Unsubmitted words sent
every pause
When pre-staged documentation already answers the question, both architectures route generation to a small, fast tier rather than a frontier model, and only escalate when multi-step reasoning or code synthesis demands it.
Live keystroke race: in-browser rules vs. network speculationType or pick a prompt below. Every typing pause runs the entry selector, source router, and Switchyard policy in your browser.
Switchyard + local index (in-browser)0.97 ms / pause
Waterfall at Enter
0 ms
Cost (3 pauses)
$0.00000
Keystrokes sent
0 chars
Selected route
light · high
Sanity 0.5s stack (TypeSafe Jev network call)340 ms / pause
Network per pause
340 ms
Cost (3 pauses)
$0.00024
Keystrokes sent
3 prefixes
Model ceiling
single small tier
Traditional waterfall (Haiku tool call after Enter)1828 ms after Enter
Methodology and caveats: our in-browser lane runs deterministic lexical, BM25 IDF, character 3-gram typo, and symptom-to-concept scoring over an inverted token posting index plus Switchyard rule routing (0.49 ms median, 1.23 ms p95, 1.77 ms p99 across N=500 runs on the full 869-section, 109-page documentation corpus; 0.24 to 2.29 ms warm server decision time on POST /docs/agent.json at commit 775c6f06, after a one-time 44.2 ms cold index build on worker start). Sanity's 340 ms figure is a hosted neural classifier call (TypeSafe Jev) and their 350 ms post-Enter figure is generative LLM time-to-first-token on Cerebras (read from sanity.io/engineering/the-0-5-second-agent). On our frozen 25-query held-out evaluation split (20 in-domain questions and 5 adversarial adjacent-domain abstentions, never tuned against), the scorer hits 22 of 25 (19 of 20 in-domain, 3 of 5 abstentions), and 35 of 35 on the in-repo regression gate (tests/docs-agent.test.mjs). When an OpenRouter key is configured on /docs/agent.json, post-Enter generation streams live LLM tokens over SSE while keeping keystroke speculation 100% local.
What does it give up?judgement: difficulty that leaves no trace in the words
A model reading the prompt can see difficulty that leaves no trace in the words, and rules over counts cannot. The labelled sets say how much that costs: 17 of 20 held-out requests went to an accepted tier, and all three misses sent a hard request one tier too cheap, multi-part professional asks with no difficulty words in them. On the 39 in-sample requests it is 39, but the weights were tuned against that set, so it is a regression gate and not evidence. Past the core properties, 12 of 12 breaks of the fast cue check and 15 of 15 of the 2026-09-29 rules turned the suite red too.
Known limitationsfrom the README
Scoring sees only what the text shows.A short question that needs deep expertise scores as a short question. The held-out misses are that shape.
Conversation state lives in memory.A restart forgets warm conversations, which costs money rather than correctness, and replicas do not share it.
Auto routes over two dialects.OpenRouter and OpenAI chat completions. Anthropic's Messages format is passed through for a caller that names a Claude model; routing it as auto would need an adapter that does not exist yet. There is no Responses API.
No per-client limits yet.No rate limits or spend caps per client key.
Jev cannot be asked again.Its answers cannot be replayed under a changed question, because the ledger holds no text to ask it with.
07 / 08The full recordPinned commit · all caveats
The full record
Every table, every judge, and every caveat.
Each part is the record as its results README and decision log state it at the pinned commit, closed until you open it.
MT-Bench, every policyeight answers per question, 2026-09-28 to 2026-09-30: the three deciding judges, then all six who graded
MT-Bench, 2026-09-28 to 2026-09-30held up, and the cost of each turn's answers
every judge's column
The table scrolls sideways.
Policy
First turns, of 80
Follow-ups, of 79
Mean
Claude
GPT
Grok, neutral
Cost
Mean
Claude
GPT
Grok, neutral
Cost
Switchyard, since 2026-09-30DeepSeek V4.1 Flash leads the light tier
66
70
69
60
$0.117
68
69
54
66
$0.215
Switchyard, before 2026-09-30gpt-6-luna led the light tier
60
52
72
47
$0.051
55
42
68
49
$0.090
Switchyard, Gemini leadingtried and not taken
59
71
50
63
$0.559
56
67
42
63
$0.819
Jev Routertypesafe/jev-router
54
56
64
41
$0.123
62
54
58
60
$0.288
Always DeepSeek V4.1 Flash, high effort
68
72
70
62
$0.098
70
71
55
65
$0.168
Always gpt-6-luna, high effort
61
51
72
48
$0.026
55
41
71
51
$0.039
Always Claude Opus 5
73
75
64
72
$1.882
66
70
49
68
$2.835
Best answer each timehindsight
80
80
80
80
$0.047
79
77
79
79
$0.091
Eight answers per question, every one graded with all eight in view. Costs are the total for each turn's answers. The follow-ups are out of 79: DeepSeek's answer to question 140 failed twice, so that question is left out for every policy, and the Claude judge's column there is out of 77. Re-grading with eight answers in view moved the figures of the six-answer run before it by up to five answers (Jev Router's follow-ups from 67 to 62), so compare within this table, not with that run's. The change costs time as well as money: DeepSeek's median answer took 6.4 s and 11.0 s, against 4.2 s and 7.0 s for gpt-6-luna at high effort. "Best answer each time" is the cheapest answer within a point of the best, which nobody can pick in advance, so it is a ceiling rather than a policy.
Every judge alone, MT-Benchheld up, graded 2026-09-30
six columns
The table scrolls sideways.
Policy
The three who decide
Added afterwards, none in the mean
All six
Claude
GPT
Grok, neutral
DeepSeek V4 Pro
Gemini 3.8 Flash
Qwen, neutral
First turns
Switchyard, since 2026-09-30
70
69
60
50
64
53
57
Switchyard, before 2026-09-30
52
72
47
34
46
31
38
Switchyard, Gemini leading
71
50
63
61
70
64
63
Jev Router
56
64
41
45
50
35
38
Always DeepSeek V4.1 Flash, high effort
72
70
62
55
67
55
59
Always gpt-6-luna, high effort
51
72
48
37
47
32
39
Always Claude Opus 5
75
64
72
72
71
75
73
Questions graded
80
80
80
77
79
80
80
Follow-ups
Switchyard, since 2026-09-30
69
54
66
52
62
54
65
Switchyard, before 2026-09-30
42
68
49
27
38
26
36
Switchyard, Gemini leading
67
42
63
50
70
58
62
Jev Router
54
58
60
44
58
45
52
Always DeepSeek V4.1 Flash, high effort
71
55
65
52
64
56
67
Always gpt-6-luna, high effort
41
71
51
26
38
26
36
Always Claude Opus 5
70
49
68
49
67
66
69
Questions graded
77
79
79
56
78
73
79
DeepSeek's and Google's models are among the answers, so their judges are interested parties; Qwen's vendor has none, so it is a second neutral judge beside Grok. Their roles were fixed before they graded, and the mean the rules were decided on did not move. DeepSeek V4 Pro was an unreliable judge on follow-ups: 56 of its 79 verdicts parsed after one retry pass, so its column rests on fewer questions than the others.
MT-Bench-101, the confirmation10 two-turn questions from each of eight tasks, drawn before any answer existed
Two-turn dialogues from MT-Bench-101 (Bai et al., ACL 2024), drawn by a seeded shuffle before any answer existed. Every model answered fresh, and each follow-up continues the dataset's own first reply, which is how that benchmark is built. The judges are the same three, with Qwen beside them and not in the mean.
MT-Bench-101, 2026-09-30held up, and the cost of each turn's answers
all five predictions held
The table scrolls sideways.
Policy
First turns, of 79
Follow-ups, of 75
Mean
Claude
GPT
Grok, neutral
Qwen, neutral
Cost
Mean
Claude
GPT
Grok, neutral
Qwen, neutral
Cost
Switchyard, since 2026-09-30DeepSeek V4.1 Flash leads the light tier
70
62
72
65
53
$0.056
73
69
67
68
58
$0.052
Switchyard, DeepSeek with reasoning offtried and not taken
58
68
53
59
62
$0.020
66
71
53
64
64
$0.015
Switchyard, before 2026-09-30gpt-6-luna led the light tier
43
24
75
27
14
$0.012
45
36
70
30
15
$0.009
Jev Routertypesafe/jev-router
34
26
70
27
16
$0.028
50
43
68
38
26
$0.033
Always Claude Opus 5
78
78
67
78
78
$1.450
68
73
48
68
73
$1.024
Here the rules before 2026-09-30 sent every question to gpt-6-luna and both DeepSeek policies sent every one to DeepSeek, so this tests which model leads the light tier and nothing about the ladder above it. One first turn is missing because a DeepSeek answer came back empty, and five follow-ups because the run's spending cap stopped it first. The dataset's own replies were references for its maths only. With its reasoning off DeepSeek answered at a median of 3.5 s and 2.6 s, against 6.9 s and 7.0 s with it on, for about a third of the cost, and held up on 12 and 7 fewer answers by the mean.
The predictions, including the one that failedeleven, each written down before any of its run's answers existed
DeepSeek leading the light tier had to hold up at least as often as Jev Router on the follow-ups, beat the rules before under two of the three judges there, and not fall behind them on first turns. It did, and its higher cost was accepted before it became the default.
The eight-answer run, MT-Bench
one failed, cheaper than predicted
HoldsAt least as many follow-ups held up as Jev Router.68 against 62.
HoldsMore follow-ups than the rules before under the Claude judge and Grok, fewer under the GPT judge.27 more, 17 more and 14 fewer.
HoldsAt least as many first turns as the rules before.66 against 60.
FailsAt least three times the rules before in cost, more than Jev Router on first turns, and within a quarter of it on follow-ups.It cost less: 2.3 and 2.4 times the rules before, and 0.96 and 0.75 times Jev Router.
HoldsGemini leading: more follow-ups than the rules before, fewer first turns.56 against 55, and 59 against 60, by one question each way.
HoldsGemini leading costs more than DeepSeek leading.$0.559 and $0.819 against $0.117 and $0.215.
The confirmation, MT-Bench-101
all five held
HoldsAt least as many as Jev Router on both turns.70 against 34, and 73 against 50.
HoldsMore than the rules before on both turns, by the mean and under Grok.70 against 43 and 73 against 45; under Grok, 65 against 27 and 68 against 30.
HoldsUnder the GPT judge, fewer than the rules before on at least one turn.The rules before lead there on both, 75 against 72 and 70 against 67.
HoldsWith its reasoning off, fewer on both turns, at under half the cost.58 and 66 against 70 and 73, at $0.020 and $0.015 against $0.056 and $0.052.
HoldsUnder Qwen, more than the rules before and Jev Router on both turns.53 against 14 and 16, and 58 against 15 and 26.
One thing changed mid-test on the six-answer run and is on the record: after 38 of the 80 follow-ups were graded, Jev Router's token cap went from 4,000 to 16,000, because DeepSeek had spent the whole cap thinking on 17 of its 74 follow-up answers and 4 of its 80 first ones. That graded the cap rather than the router, and the change could only help Jev Router. Both runs are kept, and the tables here use its answers as asked again.
What the benchmark taughtno signal we tried tells, prompt by prompt, when the cheap answer will fall short
The gap is one of prediction.
Two studies on 2026-09-29, over the same 160 graded MT-Bench prompts, graded then with six answers per question: four predictors of when gpt-6-luna's answer at high effort falls short, for $0.22, and two stricter labels to test them on, 1,125 judge calls for $5.49. The protocol and the prediction were written down before any judging, and the ones that failed are below.
Held up, of 80 by the judges' meanthe room a better router would have to win
First turns
Always gpt-6-luna, high effort62$0.026
Jev Router57$0.123
Always Claude Opus 572$1.882
The right one each time, in hindsight80$0.207
Follow-ups
Always gpt-6-luna, high effort60$0.039
Jev Router67$0.288
Always Claude Opus 566$2.847
The right one each time, in hindsight80$0.220
A router that always picked the right one of its own models would hold up on every question. On first turns the right answer each time costs more than Jev Router, and on follow-ups less. Either way it needs a signal that says, before answering, which prompts the light tier will lose. No signal tried here says it.
What ranked the first turns did not hold on the follow-ups.
Jev before answering, Jev shown luna's answer, Claude Haiku 4.5 scoring it, and gpt-6-luna scoring its own. None saw the grades or the reference answers. Thresholds were fitted on the first turns and applied unchanged to the follow-ups. A ranking score is how often a predictor rates a real failure as more likely than a success.
Ranking scores, fitted and then tested0.5 is chance
falls to chance
Each predictor, as a policyescalating to Opus on its threshold
The table scrolls sideways.
Predictor
Ranking, first turns
Ranking, follow-ups
Follow-ups held up, escalating to Opus
Cost
Jev before answering
0.80
0.57
60 of 80, never escalated
$0.043
Jev shown the answer
0.82
0.60
64 of 80
$0.380
Claude Haiku 4.5
0.67
0.57
59 of 79, never escalated
$0.167
gpt-6-luna on itself
0.56
0.58
62 of 75
$0.459
No policy built on them beat Jev Router's 67 held-up follow-ups for $0.288. Part of the reason is the label: the three judges agree on whether luna's answer held up on only 43 of the 80 first turns and 34 of the 79 follow-ups, and any two of them agree 57% to 71% of the time. A predictor cannot learn a label the judges do not agree on.
Two stricter labels, and how far the judges agree on each.
An answer key, where MT-Bench has a reference for math, reasoning and coding: each judge marked luna-high's, Opus's and Jev Router's answers right or wrong by their final result. And a pairwise label on all 160 prompts: is luna-high's answer at least as good as Opus's, asked in both orders. Kappa discounts the agreement chance alone would give: 0 is chance, 1 is perfect.
All three judges agreeof the prompts each label covers
Answer key, luna-high right or wrong50 of 55kappa 0.47
1-10 held up, the same 55 prompts33 of 55kappa 0.17
Pairwise against Opus, first turns47 of 80kappa 0.41
1-10 held up, first turns43 of 80kappa 0.20
Pairwise against Opus, follow-ups28 of 79kappa 0.14
1-10 held up, follow-ups34 of 79kappa 0.10
The answer key is cleaner than the 1 to 10 label on the same prompts, by the study's own rule of fifteen points more agreement and a higher kappa. The pairwise label is not cleaner on either turn: asking which answer is better did not remove the judges' disagreement.
Where an answer can be checked, the cheap one is right.
By the judges' majority, on the 55 prompts with a key, gpt-6-luna at high effort is right on 52, Claude Opus 5 on 52 and Jev Router on 51. The judges agree on every math and reasoning prompt; their only splits are 5 coding prompts. With so few failures there is nothing for a predictor to find.
52 of 55gpt-6-luna, high effort, right by the judges' majority28 of 29 first turns, 24 of 26 follow-ups
52 of 55Claude Opus 5, right by the judges' majority28 of 29 first turns, 24 of 26 follow-ups
51 of 55Jev Router, right by the judges' majority27 of 29 first turns, 24 of 26 follow-ups
The judges prefer Opus, and split by vendor.
Asked which answer is better, the judges together rate luna-high at least as good as Opus on 20 of 80 first turns and 27 of 80 follow-ups, even in math, where its answers are right. The 1 to 10 label's point of slack had counted 43 and 35 of those losses as holding up.
The Claude judge (Claude Sonnet 5)
First turns21 of 80
Follow-ups20 of 80
The GPT judge (GPT-6 Sol)
First turns47 of 80
Follow-ups63 of 80
Grok 4.7, the neutral judge
First turns19 of 80
Follow-ups32 of 80
How often each judge rates gpt-6-luna's answer at high effort at least as good as Opus's, in both orders. The GPT judge is the most lenient toward OpenAI's model on both turns.
No predictor finds the losses.
Against the pairwise label the four predictors rank at chance, while a judge predicting the other two reaches 0.74 on first turns and 0.67 on follow-ups. Escalating on their thresholds gains on the follow-ups what escalating as many prompts at random would.
Against the pairwise labelranking scores, and what escalating gains
at chance
The table scrolls sideways.
Predictor
Ranking, first turns
Ranking, follow-ups
Follow-ups at least as good as Opus, escalating
As many at random
Jev before answering
0.52
0.49
35 of 80
34.3
Jev shown the answer
0.61
0.40
34 of 80
34.3
Claude Haiku 4.5
0.55
0.46
27 of 79
27.0
gpt-6-luna on itself
0.54
0.53
33 of 75
32.5
gpt-6-luna at high effort alone ties or beats Opus on 27 of 80 follow-ups, so any escalation gains when it loses two comparisons in three.
A smarter per-prompt predictor built from these signals will not close the gap.
Where MT-Bench can check an answer there is no gap in right answers. On follow-ups, Jev Router's lead over luna at high effort, 67 to 60 held up, is 1 prompt on the 26 with a key and 6 on the 54 without, where "better" is the judges' call and they split by vendor.
Nothing changes in the router on the strength of this. The benchmark that would say more is one on real traffic, with answers that can be checked.
Five predictions for the label study, two of them failed.
Fixed before any judging, and printed whatever they showed.
Fixed before any judging
two failed
FailsThe answer key is far cleaner: all three judges agree on at least 85% of its prompts, against 60% for the 1 to 10 label, with a kappa of at least 0.6.Agreement is 91%, over the bar, but kappa is 0.47. All 5 splits are coding prompts.
HoldsThe pairwise label is not cleaner: within 10 points of the 1 to 10 label's agreement on each turn.The GPT judge was the most lenient on both turns, as predicted.
HoldsOn the key, luna-high is right on at least 80% of each turn's prompts.28 of 29 first turns and 24 of 26 follow-ups, so it has 2 follow-up failures, too few for the key to decide the predictor question.
HoldsOn the pairwise label no predictor separates luna-high's follow-up failures, and Jev shown the answer stays below 0.70.It scores 0.40.
FailsThe key's ceiling is above 0.85 on each turn.0.99 on first turns and 0.50 on follow-ups, a figure that rests on 2 failures.
Jev's one vote: one prompt in 56, at five times the cost.
The one rule change that reads the prompt lets Jev raise a request the rules scored at the floor, by one tier. It was scored on the 56 questions the other 2026-09-29 changes were not fitted to, and fired on 6: it helped on one coding prompt, hurt none, and changed nothing on the other five, at five times the rules' cost.
Nothing serves on the raise. It stays in the ledger as a count of what it would have raised, and the question on Jev says how that is asked.
Jev Router's answers, model by modelit picks its own model and effort per question
Jev Router chooses its own models
as it answered
It picked its model and effort per question, and on MT-Bench its answers came from gpt-6-luna, DeepSeek V4.1 Flash, Gemini 3.8 Flash and, rarely, Sonnet 5.5 or GPT-6 Sol. DeepSeek's and Google's models answered for it and for Switchyard's candidates, and xAI's, Alibaba's, DeepSeek's and Google's models judged. Every one of those vendors saw only public benchmark prompts.
First turnsMT-Bench answers
openai/gpt-6-luna43
deepseek/deepseek-v4.1-flash23
google/gemini-3.8-flash13
openai/gpt-6-sol1
Follow-upsMT-Bench answers
openai/gpt-6-luna25
deepseek/deepseek-v4.1-flash41
google/gemini-3.8-flash11
openai/gpt-6-sol1
anthropic/claude-sonnet-5.51
How every figure was madesix rules, the same for every policy
Six rulesthe same for every policy
QuestionsMT-Bench's 80 questions, first turns and follow-ups: fastchat/llm_judge/data/mt_bench/question.jsonl in lm-sys/FastChat, retrieved 2026-09-28, sha256 119565adbab82227089cefdb44c8d7e2cf04dc0a0ec233634c82e7d4e2a944f7. And MT-Bench-101's two-turn dialogues, 10 from each of eight tasks, drawn with a shuffle seeded mtbench101-confirmation-2026-09-30 from github.com/mtbench101/mt-bench-101 at bc18b3e2 (Apache-2.0), whose file has sha256 1a1e433acf7e2ef013b5001b60fd8e48bc3e4202460727e25b2ea8575aae7d80 and the sample drawn from it c5db7ce5a943bdb29481b78b4f7b85060d1ea1a03b88b75abfe189a6381708f5.
Answersgpt-6-luna at low, medium and high effort; Claude Sonnet 5 and Claude Opus 5 at low effort; DeepSeek V4.1 Flash at high effort, and on MT-Bench-101 also with its reasoning off; Gemini 3.8 Flash at high effort on MT-Bench; and Jev Router, which picks its own model and effort. All through OpenRouter, with room for 4,000 tokens, 8,000 for luna at medium, and 16,000 for luna at high, DeepSeek and Gemini. Jev Router had 4,000 on MT-Bench, where the answers that cap cut off were asked again with 16,000, and 16,000 on MT-Bench-101.
JudgesClaude Sonnet 5, GPT-6 Sol and Grok 4.7 decide, each scoring every answer to a question from 1 to 10 in one blind prompt, with the benchmark's reference answer in view where it has one. The answer order was meant to be shuffled per question and judge, and was not: later places scored up to 0.44 points lower, Jev Router lost up to 0.13 to where it sat, and gpt-6-luna at low effort sat last in nearly every prompt, at a cost not measured. Grok is the neutral judge: xAI has no model among the answers. Qwen 3.8 Max, from a vendor with none either, graded both sets beside them, and DeepSeek V4 Pro and Gemini 3.8 Flash graded MT-Bench. None of the three is in the mean.
Held upWithin one point of the best answer the tiers gave to the same prompt. Measured against the best answer rather than the strongest model's, so a prompt where the strong model stumbles cannot flatter a cheaper pick.
Follow-upsEvery answer continues the same conversation, so the answers stay comparable: on MT-Bench the first question and Opus 5's first answer, on MT-Bench-101 the dataset's own first reply. The rules decide each follow-up with their own first-turn decision as the warm conversation a minute before, as the live router would.
PoliciesEach policy is scored on the answers it would have served. The rules and Jev Router are decided per question; a decision at an effort nothing answered at is scored on the nearest answer, and the record counts it as such. The reports were written while gpt-6-luna led the light tier, so their shipped rules are the rules before 2026-09-30, and the DeepSeek candidate is what ships now.
The runs, with what each cost2,547 answers, 2,208 gradings, $39.01
The record also holds 1,125 label verdicts, 316 Jev decisions and 633 escalation predictions, which what the benchmark taught reads.
The runs in the recordbench/results/README.md
The table scrolls sideways.
Run
Date
What
Answers
Gradings
Cost
2026-09-28-turn1-four-answers
2026-09-28/29
First turns: luna, Sonnet 5 and Opus 5 at low effort, plus Jev Router; Claude and GPT judges. The first run stopped at 24 of 80 on an OpenRouter daily guardrail and a second finished it
320
160
$3.25
2026-09-29-turn1-six-answers
2026-09-29
The same questions with luna also at medium and high effort; all six answers re-graded together, then graded again by Grok. Jev Router's answers had a 4,000-token cap, which cut off 4
160 more
240
$1.75
2026-09-29-turn1-six-answers-reask
2026-09-29
The final first-turn data: Jev Router's 4 cut-off answers asked again with 16,000 tokens, and their questions re-graded by all three judges. report-with-candidate.* scores the shipped rules
4 more
12
$0.20
2026-09-29-turn2-first-measured
2026-09-29
Follow-ups as first measured: 38 of 80 before the credit ran out, and Jev Router cut off on 17 of 74 answers. Kept as the record
411
110
$3.74
2026-09-29-turn2
2026-09-29
The final follow-up data: all 80, Jev Router's cut-off answers asked again, three judges
86 more
152
$3.67
overhead-2026-09-28.md
2026-09-28
Router latency and throughput against an instant upstream
2026-09-29-escalation
2026-09-29
Four predictors of when luna's high-effort answer falls short, over the 160 graded prompts (below)
633 predictions
$0.22
2026-09-29-labels
2026-09-29
Cleaner labels from the graded answers: the answer key checked on 55 prompts, and luna-high against Opus in both orders on all 160 (below)
1,125 verdicts
$5.49
2026-09-30-turn1-vendors
2026-09-30
First turns: DeepSeek V4.1 Flash and Gemini 3.8 Flash at high effort beside the six recorded answers, which bench/regrade.mjs copied from 2026-09-29-turn1-six-answers-reask, and all eight re-graded by all three judges (below)
160 more
240
$3.06
2026-09-30-turn2-vendors
2026-09-30
Follow-ups, the same way from 2026-09-29-turn2. 79 of 80 complete: DeepSeek's answer to question 140 failed twice with an empty response
159 more
235
$4.88
2026-09-30-turn1-vendors, judges added
2026-09-30
DeepSeek V4 Pro, Gemini 3.8 Flash and Qwen 3.8 Max grade the same eight answers afterwards. Their grades follow the panel's in grades.jsonl and never enter the panel's mean (below)
236
$1.75
2026-09-30-turn2-vendors, judges added
2026-09-30
The same over the follow-ups. The $5 cap for both turns stopped the retry pass, so DeepSeek V4 Pro has 56 of 79
207
$3.09
2026-09-30-confirm-turn1
2026-09-30
The confirmation on fresh questions: 80 MT-Bench-101 first turns (2026-09-30-confirm-questions), every model answering fresh, with DeepSeek at high effort and with reasoning off, graded by the panel and Qwen 3.8 Max (below). 79 complete: one DeepSeek answer came back empty
639
316
$4.35
2026-09-30-confirm-turn2
2026-09-30
The follow-ups, each continuing the dataset's own first reply. The $3.50 cap stopped the run at 75 of 80 complete
608
300
$3.56
Not listed: one run that called no model and cost nothing, measuring a kind of request this page does not cover. The totals are the same without it.
Ten caveatsread every figure on this page with these
Ten caveats
a party's claim
The judges prefer their own vendor's answers.On MT-Bench's first turns the Claude judge put Opus 22 answers ahead of gpt-6-luna at high effort, and the GPT judge 14 behind. A claim across vendors needs the per-judge columns, not the mean alone.
Both sets are chat.80 public prompts per turn, not real traffic, and not the traffic any router here was built for.
Eighty questions per turn is a small sample.One question is 1.25 points of held up, so a gap of two or three answers is within noise.
Longer answers may be part of the gap.On MT-Bench-101 luna's answers have a median of 419 and 282 characters, against DeepSeek's 1,178 and 673, and judges tend to reward length though told not to. On the maths, graded against references, DeepSeek still scored above luna, 8.5 against 7.9 and 9.1 against 8.6 by the judges' mean score.
DeepSeek was chosen from MT-Bench's own prompts.Jev Router's DeepSeek answers did well on them, which is why it was tried, and why MT-Bench-101 was run with its predictions written down first.
Not every question is in every figure.The MT-Bench follow-ups are out of 79, after an answer that failed twice, and MT-Bench-101 has 79 first turns and 75 follow-ups, after an empty answer and a spending cap that stopped the run.
The follow-ups continue a fixed first answer.Opus's on MT-Bench, which may favour Opus continuing its own text, and the dataset's on MT-Bench-101.
Jev Router's price and model mix are its own.It chose its models itself, and on MT-Bench-101 sent most questions to gpt-6-luna: 70 of 80 first turns and 55 of 76 follow-ups.
The rules measured are the ones this repository ships.DeepSeek has led the light tier here since 2026-09-30. Switchyard's shadow deployment runs whichever release was last published to it, and which one that is is read on that server, not from the record.
Vitna built Switchyard and ran this benchmark.That makes every figure here a party's claim, which is why each judge's column is printed and the failed prediction is kept.
What was runthe commands, as the record prints them
The code is in a private repository, so these say what was run rather than something to run today. A rerun asks every model again, so its answers and scores will differ in detail from these; the runs that call models stop at their budget and resume from their own files.
The decisions at the head of this page are computed from this repository when the site is built. Every measured figure here is copied from Switchyard's benchmark record in convexityos/vitna at commit e27ca78d, committed 2026-09-30, and checked against that commit before it is published. The repository is private, so the method and the caveats are on this page rather than behind a link.
08 / 08The waitlistShadow waitlist · full record
The waitlist
Tell us what you would route.
Nothing installs yet. Leave a work email and the traffic you would point at Switchyard, and we will write when it can be run outside Convexity.