Candidates and evidence
Putting a sku in play, and recording what was measured about it.
npm run aperture:candidate:add -- --workload <id> --sku <sku> [--status watching]
npm run aperture:evidence:add -- --workload <id> --sku <sku> --kind eval --label "..." --measured-at <date> --n <int> --metric successRatePct=94.0 --source-ref <url>Evidence requires a source reference
A URL or a commit. A measurement with no provenance is a number somebody remembers, and a brief is only as checkable as the rows it cites.
--n is the sample size, recorded because a success rate over eleven runs and one over eleven thousand are not the same claim.
A metric the engine never reads is refused
Not stored and ignored. Refused.
A recorded metric nothing consumes looks like evidence and functions as decoration. Worse, it makes a workload look better evidenced than it is, which is exactly the misreading a decision register exists to prevent.
A run through the runtime can write one
Where a workload runs on a Vitna runtime, the ledger already holds what each request cost in time and tokens. This reads it back and writes one measurement per sku it served, through the same refusals a typed measurement passes.
npm run aperture:evidence:from-run -- --workload <id> --file /var/lib/vitna/ledger.jsonl --label "Harness run, 10 Sep" [--lane <alias>] [--window 2026-09-10..2026-09-10] [--dry-run]It writes latencyP90Ms, nearest-rank over the requests that were served, and n, the units served. Both rows from one run carry the same --source-ref, derived as run:<file>@sha256:<digest> over the exact rows the figures came from, so a brief that weighs the two can say they were observed together. Each sku's row is labelled with the run's label and its own id, and the whole run enters in one transaction or not at all.
It also prints what a unit cost, which is the observed tokens spent against the price record rather than read off a vendor's bill, and the `workload:edit` line that enters the profile it observed.
It measures, and it does not grade
Vitna is not an eval platform, and this is where that boundary is load-bearing rather than a slogan. A run yields a latency and a sample size because those are things the ledger observed. It never yields successRatePct.
The tempting number is the share of requests that came back, and it is the wrong number: a refused or failed request is a request nobody served, not a wrong answer. An outage and a bad answer are not the same finding and only one of them is a quality problem. So refusals and failures are counted, written on the row in words, and never turned into a metric, and a sku that served nothing at all yields no measurement rather than a rate of zero.
successRatePct keeps arriving from wherever the team already measures quality.
Importing from CI eval harnesses
Teams already running Promptfoo, Braintrust, or UK AISI Inspect AI in CI can POST raw JSON outputs directly to CI evaluation evidence with adapter set to promptfoo, braintrust, or inspect, alongside vitna for native payloads. Every adapter funnelled through evidence.add enforces the same metric whitelist (successRatePct, latencyP90Ms, edgeCaseFailures, verifiedCapabilities), requires a provenance source reference, rejects future dates, and keeps append-only label uniqueness (409 duplicate_evidence on a re-submitted label).
Evidence is append-only
There is no edit and no delete. A measurement that turned out to be wrong is superseded by a later one, and both stay, because which one the engine read at the time is part of what makes a past decision checkable.
