Get started

Stateless rollout watchdog

Source

Evaluate standing-decision drift, shadow rule promotion readiness, and routed canary health in memory against Witness list prices, provider compliance terms, and three-provenance SKU capabilities, with no API key and zero retention.

text
POST /api/v1/watchdog
bash
curl -X POST https://vitna.ai/api/v1/watchdog \
  -H "content-type: application/json" \
  -d '{
    "workload": {
      "id": "ci/deep-research",
      "classId": "model",
      "incumbentSkuId": "openai/gpt-5.4",
      "unitProfile": { "inputTokensPerUnit": 4000, "outputTokensPerUnit": 800 },
      "constraints": {
        "qualityFloor": 93.0,
        "latencyP90Ms": 2500,
        "requiresCapabilities": ["tools"]
      }
    },
    "standingDecision": {
      "id": 12,
      "outcome": "stay",
      "decidedAt": "2026-10-20T00:00:00.000Z",
      "recordedUnitCostUsd": 0.018
    },
    "rules": [
      {
        "id": 4,
        "position": 1,
        "mode": "route",
        "kind": "traffic_share",
        "share": 5,
        "serveSkuId": "anthropic/claude-sonnet-4-6"
      }
    ],
    "ledgerRows": [
      { "at": "2026-10-24T10:00:00Z", "outcome": "served", "serve": "rule:1", "sku": "anthropic/claude-sonnet-4-6", "cohort": 2, "latencyMs": 3100 }
    ],
    "evidence": [
      { "skuId": "openai/gpt-5.4", "measuredAt": "2026-10-22", "metrics": { "successRatePct": 95.0 } },
      { "skuId": "anthropic/claude-sonnet-4-6", "measuredAt": "2026-10-22", "metrics": { "successRatePct": 94.5 } }
    ]
  }'

What is checked

Runs `evaluateWorkloadWatchdog` in memory over the inline workload, optional standingDecision, rules, fallbacks, observations, ledgerRows, evidence, and thresholds:

  1. Standing decision drift: unit-cost movement against standingDecision.recordedUnitCostUsd, provider compliance terms, three-provenance capabilities, evaluation qualityFloor and edgeCaseVeto, fallback reliance, unserved attempt rate, and latencyP90Ms.
  2. Per-rule canary ladder: evaluates each shadow or route rule for promotion readiness (promote_ready), rung expansion (expand_or_adopt_ready across 5% -> 25% -> 50% -> adopt_incumbent), shadow retirement (retire_shadow), stale decision authority (stale_authority), or immediate canary rollback (rollback_recommended), returning exact operator commands (recommendedCommands).

Payloads are never persisted (Cache-Control: no-store). Also exposed to MCP clients as vitna_check_rollout on `POST /api/v1/mcp`.

Response shape

json
{
  "workload": {
    "id": "ci/deep-research",
    "name": "Inline workload",
    "classId": "model",
    "servingSkuId": "openai/gpt-5.4",
    "incumbentSkuId": "openai/gpt-5.4",
    "policyAlias": null
  },
  "verdict": "rollback_recommended",
  "standingDecision": {
    "id": 12,
    "outcome": "stay",
    "decidedAt": "2026-10-20T00:00:00.000Z",
    "title": null,
    "recordedUnitCostUsd": 0.018,
    "currentUnitCostUsd": 0.018
  },
  "decisionChecks": [ ... ],
  "driftReasons": [],
  "observations": {
    "readingsWindow": "2026-10-24..2026-10-24",
    "servedUnits": 1,
    "fallbackSharePct": 0,
    "unservedRatePct": 0
  },
  "rules": [
    {
      "id": 4,
      "position": 1,
      "ruleKey": "rule:1",
      "kind": "traffic_share",
      "mode": "route",
      "share": 5,
      "serveSkuId": "anthropic/claude-sonnet-4-6",
      "verdict": "rollback_recommended",
      "reason": "Routed canary rule 4 (anthropic/claude-sonnet-4-6) breached safety bounds: observed latency 3100ms exceeded 2500ms. Remove the rule and re-emit policy for ci.",
      "breaches": ["observed latency 3100ms exceeded 2500ms"],
      "telemetry": { ... },
      "recommendedCommand": "npm run aperture:rule:remove -- 4 --reason \"Canary rollback on anthropic/claude-sonnet-4-6: observed latency 3100ms exceeded 2500ms\""
    }
  ],
  "thresholds": {
    "minShadowMatches": 50,
    "minCanaryUnits": 100,
    "maxUnservedRatePct": 5,
    "maxFallbackSharePct": 10,
    "maxPriceDriftPct": 10
  },
  "recommendedCommands": [ ... ],
  "epistemicNotice": "Unserved attempt rates reflect transport or provider availability only; answer quality (successRatePct) enters exclusively from recorded workload evaluations.",
  "sentence": "ci/deep-research: rollback recommended on 1 routed rule (rule 4 -> anthropic/claude-sonnet-4-6).",
  "ledgerSummary": { ... },
  "privacy": "Evaluated in memory with zero retention; inline workload profiles, rollout rules, ledger rows, and evaluation figures are never stored.",
  "meta": {
    "generatedAt": "2026-11-05T12:00:00.000Z",
    "source": "snapshot",
    "ruleCount": 1
  }
}