> Source: https://vitna.ai/docs/routing/router-challenge > Raw Markdown: https://vitna.ai/docs/routing/router-challenge.md > Corpus index: https://vitna.ai/llms.txt # Router Challenge (#72) A router executes a routing policy per request. Vitna determines whether handing the choice to a router beats naming a model. ## The Epistemic Problem of Pricing a Router Every fixed model or provider infrastructure class has a published list price on Witness (`witness.price_period`). Vitna archives the raw bytes, timestamps the read, and tracks the price period. A dynamic router (such as Not Diamond, Martian, RouteLLM, or OpenRouter Auto) has no single published list price: 1. It routes dynamically across an arbitrary set of underlying models. 2. It charges in one of several ways: nothing on top of the selected model's own rate, which [OpenRouter's Auto Router documentation](https://openrouter.ai/docs/guides/routing/routers/auto-router) states for that router (retrieved 2026-08-26), a fee per request, or a percentage markup. 3. Its effective cost per unit is knowable only retrospectively from observed deployments or controlled test runs. A decision engine that assumes a static list price for a router would be inventing a figure the market does not publish. Aperture solves this through Router Challenge (#72): evaluating a router candidate retrospectively from confirmed telemetry recorded in `aperture.deployment_observation` and evals recorded in `aperture.evidence`. The [/challenge](https://vitna.ai/challenge) page runs the same function over three illustrative scenarios. Nothing on it was observed, and it names no router and no model. ## The Economics of a Router Candidate For a window of observed requests, the total router cost is: ``` Total Spend = Underlying Token Spend + Router Surcharges Router Unit Cost = Total Spend / Observed Units ``` When a router charges a fee per request, the platform fee can consume a substantial fraction of the token savings gained by picking cheaper models. Router Challenge computes the fee drag: ``` Fee Drag = Router Surcharge per Unit / (Incumbent Token Cost - Router Underlying Token Cost) ``` If underlying model selection saves \$0.006 per unit, but the router fee is \$0.004 per unit, the fee drag is 66.7%, leaving only \$0.002 in net savings. Like every decision in Aperture, candidates are compared on cost per successful workload unit: ``` Successful Unit Cost = Unit Cost / Task Success Rate ``` A router that cuts token cost in half but drops task accuracy from 95% to 85% may produce a higher cost per successful unit once failed retries are factored in. ## Running a Router Challenge From the command line: ```bash npm run aperture:challenge -- <workload-id> --router <router-sku> [--fixed <sku>] [--surcharge <usd>] [--format markdown|json] ``` ## Router ids the engine knows `KNOWN_ROUTERS` in `platform/engine/router-candidate.mjs` carries a default fee for four router ids, used when `--surcharge` is not given. One of those defaults is a sourced fact. The other three are the engine's own assumptions, so a challenge run against one of them should pass `--surcharge` with the fee your own contract states. | Router identifier | Default fee | Source | |---|---|---| | `openrouter/auto` | \$0.000 / req | [OpenRouter's Auto Router documentation](https://openrouter.ai/docs/guides/routing/routers/auto-router), retrieved 2026-08-26: you pay the selected model's standard rate, and nothing more for the routing | | `notdiamond/custom-router` | \$0.001 / req | Not sourced. An engine default, never read from Not Diamond | | `martian/model-router` | \$0.004 / req | Not sourced. An engine default, never read from Martian | | `routellm/mf` | \$0.000 / req | Not sourced. An engine default | ## Minimum Evidence Guardrail A router candidate evaluation requires at least 20 observed units on the workload (`MIN_ROUTER_OBSERVATIONS = 20`). Aperture refuses to evaluate a router on thinner data. ## Verdicts * **SWITCH:** The router satisfies the quality floor, passes all edge case vetoes, and beats the incumbent by at least the workload's `economicThresholdPct` on cost per successful unit. * **STAY:** The fixed incumbent model remains more economical, or the router fails the quality floor. * **TEST:** The router achieved favorable economics, but triggered edge case vetoes or has marginal sample size requiring targeted validation before production rollout.