Get started

Multi-class unit profiles and compound pipelines

Source

Evaluate end-to-end AI workflows that fan out across multiple component classes, attribute runtime x-vitna-run trace volume back to each stage, and separate decision-ready migrations from exploratory class-peer levers.

Multi-class unit profiles across all 10 Witness classes

Production AI workloads consume more than language model tokens. UNIT_TERMS in platform/engine/decide.mjs maps every one of the 33 registered Witness price dimensions across all 10 component classes into unitProfile keys:

ClassProfile keysWitness dimensions
modelinputTokensPerUnit, cachedInputTokensPerUnit, cacheWrite5mTokensPerUnit, cacheWrite1hTokensPerUnit, outputTokensPerUnit, batchInputTokensPerUnit, batchOutputTokensPerUnit, priorityInputTokensPerUnit, priorityOutputTokensPerUnit, flexInputTokensPerUnit, flexOutputTokensPerUnit, outputImagesPerUnit, batchOutputImagesPerUnitinput_per_mtok, cached_input_per_mtok, cache_write_5m_per_mtok, cache_write_1h_per_mtok, output_per_mtok, batch_input_per_mtok, batch_output_per_mtok, priority_input_per_mtok, priority_output_per_mtok, flex_input_per_mtok, flex_output_per_mtok, output_per_image, batch_output_per_image
computegpuHoursPerUnitgpu_hour_on_demand
retrievalembedTokensPerUnit, rerankTokensPerUnitembed_per_mtok, rerank_per_mtok
speechaudioMinutesBatchPerUnit, audioMinutesStreamingPerUnit, ttsCharsPerUnitaudio_minute_batch, audio_minute_streaming, tts_per_mchar
toolingtoolCallsPerUnit, pageFetchesPerUnit, containerHoursPerUnit, sessionHoursPerUnit, storageGbDaysPerUnittool_call_per_1k, page_fetch_per_1k, container_hour, session_hour, storage_per_gb_day
extractionocrPagesPerUnit, layoutPagesPerUnitpage_ocr, page_layout
translationtranslateInputCharsPerUnit, translateOutputCharsPerUnittranslate_input_per_mchar, translate_output_per_mchar
imageimagesPerUnit, batchImagesPerUnit, imageMegapixelsPerUnitimage_per_image, batch_image_per_image, image_per_megapixel
videovideoSecondsPerUnitvideo_per_second
musicmusicTracksPerUnitmusic_per_track

Because register.mjs, reconcile.mjs, stackfile.mjs, decide(), and evaluatePipeline() all read from that single UNIT_TERMS table, non-model workloads participate in the exact same decision briefs, invoice reconciliations, counterfactuals, and signed policy workflows as language models.

Compound pipeline Bill of Materials

Evaluating stages one lane at a time hides two structural properties of real agentic and multimodal systems:

  • Stage fan-out dominates unit price: a web-search stage called 8 times and a layout-extraction stage parsing 15 pages per report frequently cost more per end-user run than the final synthesis model called once.
  • Sequential reliability compounds: across critical-path stages with recorded successRatePct evaluations, compound pass rate is the product of stage pass rates, raising costPerSuccessfulRunUsd above raw list costPerRunUsd.

evaluatePipeline (platform/engine/pipeline.mjs) computes:

  • Per-stage unitCostUsd, invocationsPerRun, runCostUsd, monthlyCostUsd, costSharePct, compliance/capability eligibility, quality, and p90 latency.
  • Spend rollups breakdown.byClass and breakdown.byProvider, plus explicit breakdown.provenance splitting Witness archive observations (witnessedRunCostUsd) from customer-entered private infrastructure rates (enteredRunCostUsd).
  • Pipeline bottlenecks (dominantCostStage, dominantCostClass, dominantLatencyStage, and weakestQualityStage).

Runtime trace attribution (x-vitna-run)

When callers pass x-vitna-run (and optional x-vitna-agent) headers through vitna-runtime, every JSONL ledger line carries runId and agentId. summarizeLedgerForPipeline groups ledger rows by runId to measure empirical upstream invocationsPerRun per lane, local response cache hits (cacheHits), clean vs fallback-degraded vs failed runs (runCompletionRatePct), per-run call distributions, and per-lane p50/p90/p95 latencies.

When paired with evaluatePipeline, empirical runtime_ledger fan-out replaces static stage guesses and reports invocationDriftPct against declared fan-out.

Verified switches vs exploratory peer levers

evaluatePipeline keeps two strictly separated optimization buckets:

  • `decisionReadySwitches`: stages where a candidate clears compliance terms, three-provenance capabilities, qualityFloor, edgeCaseVeto, and economicThresholdPct inside decide(), returning outcome: "switch". Only decisionReadySwitches lower optimization.optimizedCostPerRunUsd.
  • `exploratoryTests`: eligible candidates (including active Witness class peers discovered when discoverPeers: true) that cost less at list price while still awaiting workload successRatePct evaluations or critical-failure clearance. Exploratory levers carry zero quality claim and emit exact aperture:candidate:add and aperture:evidence:add commands.

Evaluate compound pipelines from the CLI with `aperture:pipeline`, for an authenticated organisation via `GET /api/v1/org/pipeline`, or statelesly via `POST /api/v1/pipeline` and MCP vitna_analyze_pipeline.