Get started

Router challenge

Source

Evaluate in memory whether handing model selection to a dynamic router (openrouter/auto, martian/model-router, notdiamond/custom-router, routellm/mf, or a custom router) beats naming a fixed SKU, with zero retention. The HTTP surface for Router Challenge (#72).

text
POST /api/v1/router-challenge
bash
curl -X POST https://vitna.ai/api/v1/router-challenge \
  -H "Content-Type: application/json" \
  -d '{
    "workload": {
      "id": "acme/support-triage",
      "name": "Support triage",
      "unitLabel": "tickets",
      "constraints": {
        "qualityFloor": 88.0,
        "economicThresholdPct": 15.0,
        "edgeCaseVeto": ["pii_redaction"]
      }
    },
    "incumbent": {
      "skuId": "openai/gpt-5.6-luna",
      "name": "GPT-5.6 Luna",
      "unitCostUsd": 0.03,
      "successRatePct": 90.0,
      "latencyP90Ms": 1200
    },
    "router": {
      "skuId": "martian/model-router",
      "surchargePerRequestUsd": 0.004
    },
    "observations": [
      { "units": 500, "tokenSpendUsd": 7.50, "latencyP50Ms": 450 }
    ],
    "routerEvidence": {
      "successRatePct": 92.0,
      "n": 500,
      "latencyP90Ms": 1400,
      "edgeCaseFailures": []
    }
  }'

Response

json
{
  "status": "evaluated",
  "workloadId": "acme/support-triage",
  "workloadName": "Support triage",
  "routerSkuId": "martian/model-router",
  "routerName": "Martian Model Router",
  "outcome": "switch",
  "reason": "Martian Model Router beats incumbent GPT-5.6 Luna by 31.1% on cost per successful unit while meeting the 88.0% quality floor.",
  "checks": [
    {
      "name": "quality floor",
      "threshold": "88.0%",
      "observed": "92.0%",
      "passed": true
    }
  ],
  "knownRouters": { ... },
  "meta": {
    "generatedAt": "2026-09-12T09:14:02.881Z",
    "minRequiredUnits": 20
  }
}

Epistemic refusal below 20 units

A dynamic router publishes no static per-token list price, so its effective unit cost is derived from observed trial telemetry plus any per-request or percentage routing fee. When observations sum to fewer than 20 units, the endpoint returns 200 with status: "refused", outcome: "stay", and the exact observation shortfall rather than extrapolating from a handful of prompts.

Zero retention

Payloads are evaluated strictly in memory and are never written to the database, disk, or application logs.

Caching

no-store.