> Source: https://vitna.ai/docs/runtime/serving
> Raw Markdown: https://vitna.ai/docs/runtime/serving.md
> Corpus index: https://vitna.ai/llms.txt

# Serving a lane

Starting a runtime against a signed policy, the two request dialects it answers,
and how a lane reaches its model, fails over and stays inside the provider
allowlist.

## Starting it

```bash
VITNA_POLICY=/etc/vitna/policy.json \
VITNA_PUBLIC_KEY=$(cat /etc/vitna/policy-key.pub) \
VITNA_LEDGER=/var/lib/vitna/ledger.jsonl \
ANTHROPIC_API_KEY=... OPENAI_API_KEY=... \
vitna-runtime
```

`vitna-runtime` is on the path once `npm install -g vitna-runtime` has run, on
Node 22.13 or later. [Installing the runtime](installing.md) covers npm, the
container, and where the policy and its key come from.

| Variable | Meaning |
|---|---|
| `VITNA_POLICY` | Path to a signed policy on this machine. Required, unless the document arrives encoded. |
| `VITNA_POLICY_B64` | The same document, base64 encoded, for a platform where secrets are set and no file can be copied. Exactly one of the two, and both is refused at boot. |
| `VITNA_PUBLIC_KEY` | The key to verify it against, pinned once out of band. Required. Several, comma separated, are accepted, which is how a signing key is replaced without an outage: a policy signed by any of them verifies. |
| `VITNA_LEDGER` | Where to append one line per request. Optional, and its absence is said out loud. |
| `VITNA_PORT` | Default 8787. |
| `VITNA_HOST` | Where to listen. Default `127.0.0.1`, where no key is asked for. Anything else needs `VITNA_INBOUND_KEY`, or the runtime refuses to start. |
| `VITNA_INBOUND_KEY` | The bearer key every `/v1` request and the full `/status` must send. See [the front door](#the-front-door). |
| `VITNA_UPSTREAM_TIMEOUT_MS` | The longest a completion that is not streamed may take. Default 540000. See [when a provider hangs](#when-a-provider-hangs). |
| `VITNA_UPSTREAM_FIRST_BYTE_MS` | The longest a stream may take to send its first chunk. Default 300000. |
| `VITNA_UPSTREAM_IDLE_MS` | The longest a stream may go quiet between chunks. Default 300000. |
| `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `OPENROUTER_API_KEY` | Your own keys, for the providers your lanes name. |
| `VITNA_ROUTE_<PROVIDER>` | `direct` or `openrouter`, to force a route. |
| `VITNA_ALLOW_PROVIDERS` | Comma-separated. Unset is every provider. |

## Operator note: starting it from a checkout

This section is for Vitna's own operators. The repository is private, so a
reader outside Vitna has no checkout to run it from and starts the runtime from
npm as above. In a checkout the same entry point runs as a script, with the
same variables:

```bash
VITNA_POLICY=/etc/vitna/policy.json \
VITNA_PUBLIC_KEY=$(cat /etc/vitna/policy-key.pub) \
VITNA_LEDGER=/var/lib/vitna/ledger.jsonl \
ANTHROPIC_API_KEY=... OPENAI_API_KEY=... \
npm run runtime:start
```

## Why the key is pinned rather than read from the file

A policy carries the public key that signed it, so a reader can see which key it
was. Checking a signature against the key that came with it proves only that the
file agrees with itself, which anyone who can write a file can arrange.

So the runtime is given the key once, out of band, and refuses to start without
one. The key to pin is the one `/console/deployment` prints beside the signed
file, not the key over the daily roots on [/witness](https://vitna.ai/witness),
which signs the record and not your policy;
[Installing the runtime](installing.md) says where both come from. That is
what "refuses to start unsigned" has to mean to be worth anything.

## The front door

The runtime holds your provider keys, so anybody who can reach it can spend
them. Since 0.2.0 it decides who can reach it rather than leaving that to
whatever is in front of it:

- **It listens on `127.0.0.1` by default**, where only the machine itself can
  reach it and no key is asked for. A runtime beside the service that calls it
  needs nothing new, and neither does the operators' `runtime:local`.
- **Without a key, it trusts the machine's processes, not the machine's
  browser.** A web page open on the same machine can reach `127.0.0.1` too. So a
  runtime with no key answers 403, with no ledger row, to a request whose `Host`
  is not a loopback name (a DNS-rebinding page naming its own host), and to a
  `/v1` request that carries an `Origin` header or whose body is not sent as
  `content-type: application/json` (a page skipping the CORS preflight). An SDK
  sends none of those. To call the runtime from a browser, set a key.
- **Anywhere else, it refuses to start without `VITNA_INBOUND_KEY`.** Set
  `VITNA_HOST=0.0.0.0` with no key and the boot names both variables and stops.
  The container image and the Fly recipe set that host, so both need the key.
- **With a key set, every `/v1` request sends it** as
  `authorization: Bearer <key>`, compared in constant time. That is the header
  an OpenAI SDK sends its API key in, so the key goes where the SDK's key goes.
  An Anthropic SDK sends its API key as `x-api-key`, which is not read here: give
  it the key as an auth token instead (`ANTHROPIC_AUTH_TOKEN`, or `authToken` in
  code), which it sends as a bearer. A request without the key is answered 401
  before its body is read, and writes no ledger row, because the ledger is your
  file and a stranger should not be able to append to it.
- **A health check needs no key.** `GET /status` with no header answers
  `{ "ok": true, "version": "0.2.0" }` and nothing else, and so does
  `GET /healthz` whatever it sends. The full page, with the org, the lanes, the
  decision ids and which keys are set, answers only with the key. A request that
  sends the wrong key to `/status` is told 401 rather than handed the short body,
  because it meant to authenticate.

**What a bearer key is.** It stops the anonymous caller and nothing more. It is
one key for every client, so it cannot say which client sent a request, and
rotating it means replacing it everywhere it is used. Per-client keys and mutual
TLS wait for a customer who needs them.

## The request

The runtime speaks the OpenAI chat completions dialect, because that is the one
every SDK already speaks. Point a base URL at it and name a lane:

```bash
POST /v1/chat/completions
{ "model": "vitna/deep-research", "messages": [{ "role": "user", "content": "..." }] }
```

`/chat/completions` answers the same, so a base URL that already ends in `/v1`
works too.

**A model name without the `vitna/` prefix is passed through untouched.** It is
marked unrouted on the ledger and attributed to no decision, because nothing in
the record chose it. That is what lets a team point one service at the runtime
and move its lanes over one at a time rather than all at once.

**Every answer says how it was routed**, in four response headers holding the
same values as the [ledger](ledger.md) row of the attempt that answered:

| Header | What it holds |
|---|---|
| `x-vitna-lane` | The lane the request named |
| `x-vitna-sku` | The sku that answered, which is the fallback's where a fallback did. The one to price the answer at |
| `x-vitna-decision` | The decision that authorized that sku |
| `x-vitna-policy` | The digest of the signed policy that carried the decision |

A header is left off where the row holds nothing: an unrouted name sends only
`x-vitna-policy`, and a lane the policy does not carry sends its name and no sku
or decision. A stream sends them with its first byte. A request refused at the
[front door](#the-front-door) is told none of them.

**A caller that keeps its own records can name each request.** Send
`x-vitna-request-id` and every ledger row the request writes carries it as
`requestId`, each fallback attempt included, so the two records join on it
rather than on time and lane. It is recorded as sent, trimmed: 1 to 128
printable ASCII characters with no spaces. Anything else is refused with a 400
and a row, because a caller that sent an id is relying on the join, and a row it
cannot find is one it would count as missing. An empty header is no id.

**It can also say who the request was for.** An agent or a routine that calls
models through the runtime sends its own id in `x-vitna-agent`, and the run it
is in, in `x-vitna-run`. Every row the request writes carries them as `agentId`
and `runId`, so what one agent's runs cost totals from the ledger rather than
being worked out from times and lanes. Both follow the request id's rule, and a
run with no agent is refused, since a run is always some agent's. Neither is
checked against anything: they are what the caller says, as the request id is.
None of the three is sent to a provider.

## The other dialect

Some clients speak only Anthropic's messages API, coding harnesses among them.
They reach the same lanes at the same base URL:

```bash
POST /v1/messages
{ "model": "vitna/deep-research", "max_tokens": 1024,
  "messages": [{ "role": "user", "content": "..." }] }
```

`/messages` answers the same, for a base URL that already ends in `/v1`.

**A request that names a model rather than a lane is not translated.** Where
`ANTHROPIC_API_KEY` is set, and Anthropic is neither outside
`VITNA_ALLOW_PROVIDERS` nor forced through OpenRouter, it goes to Anthropic as it
was written, with the caller's `anthropic-version` and `anthropic-beta` headers
and nothing else of the caller's, its credential least of all. The answer comes
back as Anthropic sent it, a stream byte for byte, so thinking, prompt caching and
Anthropic's own errors reach the client as they would direct, and the row is
marked unrouted like any other model name. Nothing routed that request, so there
was nothing to translate it for, and the translation below would only lose the
things such a client leans on. Otherwise it is translated and sent where any
unrouted name goes, as before.

For a lane, it is a translation and not a second runtime. The request is read as a
chat completion, routed by the one path that routes, and the answer is written
back in the dialect it arrived in, so a lane, a chain, a rule, a shadow match and
a [ledger](ledger.md) row are identical whichever endpoint asked. Nothing on the
row says which dialect did: a lane's units are its units, and totalling harness
traffic separately would be reporting on the client rather than on the lane.

**Tools are translated, on every route.** Tool definitions, tool choice, a tool
call coming back and a tool result going out all carry, including through a
stream, where a call's arguments arrive as the fragments each dialect streams
them in. That holds whether the lane is served through OpenRouter or by Anthropic
directly, so a harness works without anybody changing which of their accounts
serves their traffic.

A `system` message inside the messages array is carried too, and stays where it
was sent rather than joining the leading system prompt, because a system message
the model reads in a different place is a different prompt. This dialect's own
documentation does not describe one; Claude Code sends one, which is the sort of
thing only running a harness finds.

Two fields are carried by callers and deliberately not forwarded, and they are
the only two. `cache_control` marks a prompt prefix for Anthropic's own prompt
cache; it cannot change a token of the answer, only what the answer costs, and
the ledger records the tokens the provider actually reported either way.
`is_error` marks a tool result as a failure; its content is forwarded verbatim,
so the model still reads the error the tool produced, and the other dialect has
nowhere to put the flag.

Everything else a caller sends and this runtime does not translate is refused by
name, with what to do instead. The ones worth knowing before you start:

| Sent | Answer |
|---|---|
| `thinking` | Refused. A thinking budget has no portable spelling in the dialect the runtime forwards, so turn extended thinking off for the lane. Claude Code sends this unless `MAX_THINKING_TOKENS` is `0`, which is why the launcher sets it. |
| `context_management` | Refused. It edits the turn inside Anthropic's API and a routed lane has no equivalent step. Claude Code sends it only to manage thinking blocks, so it goes when thinking does. |
| `output_config` | **Carried.** Its `effort` is a level, and the dialect this runtime forwards has the same three under the name `reasoning_effort`. A level outside `low`, `medium` and `high` is refused rather than rounded, and any other key of that object is refused by name. |
| `top_k` | Refused. No counterpart, and rescaling it onto `top_p` would be an invented mapping. |
| A server tool (`web_search`, `code_execution`, a container) | Refused by name. Anthropic's own API serves those rather than the model, so a lane cannot reach one whatever it is routed through. |
| `mcp_servers` | Refused. Connect the server to your harness instead, where its tools arrive as ordinary tools. |
| An image inside a `tool_result` | Refused. A tool message in the other dialect carries text and nothing else, and moving the image into a following user turn would put it where the model reads the user saying it. |
| `POST /v1/messages/count_tokens` | 404, named. Counting needs the tokenizer of whichever model the lane serves, and an estimate would be a figure nobody measured. |

Those refusals are the translation's, so they meet a request naming a lane. A
request naming a model, with Anthropic reachable directly, goes as it was
written, as above, thinking and all.

A harness is pointed at a lane with its own environment: the base URL of this
runtime, a key, and `vitna/<alias>` where it expects a model name. The key is
never a provider key: on a runtime with no `VITNA_INBOUND_KEY` any non-empty
value does, and on one with a key set it is that key, sent as a bearer.
[Putting a coding harness on a lane](harness.md) is the command that does it,
and covers the two things that go wrong when it is done by hand.

## How a lane reaches its model

Vitna decides *what model*. Which of your accounts serves it is your own
operational fact, so it comes from the environment rather than from the record:

- A provider with a direct key and a direct adapter goes direct. The runtime
  speaks `openai` and `anthropic` directly.
- Anything else rides OpenRouter, which reaches the rest through one dialect.
- `VITNA_ROUTE_ANTHROPIC=openrouter` forces the substrate;
  `VITNA_ROUTE_ANTHROPIC=direct` forces the direct route.

A forced route that cannot be taken is unreachable rather than quietly becoming
the other one. An operator who wrote down `direct` has said something about
committed spend or data handling, and a silent reroute would break it.

The name sent differs by route and is never derived: Vitna registers
`claude-opus-4-8` and OpenRouter lists `anthropic/claude-opus-4.8`. Both are
signed into the lane, so this is a lookup rather than a rule.

## Failing over

A lane with a [chain](../routing/fallbacks.md) behind it walks that chain when
the primary cannot serve the request now: a transport failure, a 429, or the
provider's own 5xx.

Everything else stops at the first attempt. A 4xx is the caller's request being
wrong, and asking a second model the same wrong question spends money to be told
the same thing twice. **A 401 is deliberately on that side of the line**: a bad
key is a configuration error, and failing over would serve from the chain
indefinitely while the real fault stayed invisible.

**One 4xx fails over, by name: Anthropic's empty account.** OpenAI reports an
account with no credit as a 429, which fails over by its status. Anthropic
reports one as a 400 ("Your credit balance is too low to access the Anthropic
API"), which would otherwise stop the chain at exactly the moment a fallback is
for. So a 400 from Anthropic, reached directly, whose message says "credit
balance" or "billing" walks the chain, and its [ledger](ledger.md) row is a
`failed` one whose `error` starts by naming it (`credit_exhausted`) before the
provider's own words. Every other 400 still stops at the first attempt,
including a malformed request's, which says neither.

A budget that runs out before anything reached the caller fails over too, as
[a hang](#when-a-provider-hangs) should. A caller who has hung up never does.

There is no retry of the same model. A second attempt is a second serve and gets
its own ledger row, so a chain of one is a chain of one.

An exhausted chain returns the last failure. Nothing is invented in its place.

## The provider allowlist

`VITNA_ALLOW_PROVIDERS=anthropic,openai` is a boundary, and it binds fallbacks
exactly as it binds a primary: a provider outside it is unreachable by every
route, including the substrate, including as the last entry of a chain. An
allowlist a chain could reach past is not an allowlist.

Unset means every provider, which is the right default for a gateway nobody has
stated a boundary for. Inventing a restrictive one would break lanes for a
compliance posture the operator never described.

**A model name sent without a lane is held to it too, by its route.** A lane is
checked against the provider whose model its signed policy names. A name with no
`vitna/` prefix carries no such provider, and the runtime does not guess one from
the name's shape, so the list is read against where the request would go:

| Route an unrouted name takes | Needs listed | Because |
|---|---|---|
| OpenRouter, where `OPENROUTER_API_KEY` is set | `openrouter` | Behind a name sent there can be any author's model on any host, so listing a model's own provider cannot admit it |
| OpenAI directly, where `OPENAI_API_KEY` is set | `openai` | OpenAI is who sees the request |
| Anthropic directly, for a `/v1/messages` request [sent as written](#the-other-dialect) | `anthropic` | Anthropic is who sees it |

A route the list does not admit is passed over, so with both keys set and
`VITNA_ALLOW_PROVIDERS=anthropic,openai`, a name goes to OpenAI rather than through
OpenRouter. With no admitted route the request is refused with a 422 naming the
variable and what to list, and a ledger row, as every configuration refusal is.
Listing `openrouter` admits no lane that was not admitted before. Until
2026-09-29 the list was not read for these names at all, so a name with no
prefix reached OpenAI or OpenRouter whatever it said.

The boot lines say which key an unrouted name will spend, and a key the list
leaves nothing to carry says so, rather than that a fallback might reach it. An
OpenAI or Anthropic key carries nothing once its provider is off the list. The
OpenRouter key is different, since a lane is held to its model's provider and
not its route: leaving `openrouter` off shuts out only names sent without a
lane, so the key carries nothing only when every provider the list admits is
reached some other way, directly for instance.

## What each lane has done

`/status` carries a per-lane count of what has been served, refused, failed and
failed over since the process started, with the tokens each lane spent.

**It is read, never enforced.** Nothing is refused for exceeding anything,
because a gateway that starts dropping production traffic at month end is a page
at 2am that Vitna wrote. Enforcement that drops traffic is autopilot-shaped and
waits for an explicit opt-in a customer decides.

Nothing is counted in money either. A cost needs a price, and a price frozen
into a signed policy at emit time is exactly the stale figure presented as
current that the record exists to prevent. The units and the tokens are here;
the tape prices them later, at the right dates.

## Rules

A lane may carry [rules](../routing/rules.md): declared predicates on what
the runtime can count about a request, evaluated in order, first match
deciding. A SHADOW rule routes nothing and stamps the ledger row of what
actually ran with what it would have served. A ROUTED rule's target leads the
attempts, attributed to the decision that promoted it, with the lane's own
order standing behind it, because serving the decided model is always
authorized.

What a rule may see is counts of what will be forwarded, never its meaning:
`context_chars_over` compares the summed length of the request's string
content against a threshold, exactly, and no token estimate is ever made.

## Streaming

`"stream": true` works on both endpoints. A same-dialect route is forwarded byte
for byte. An Anthropic-direct lane is translated into chat completion chunks, and
the usage, which arrives split across two events in that dialect, is held and
recorded together.

A tool call streams through that translation as well, which takes one thing that
cannot be derived either way: the two dialects number things differently. Every
content block counts in one, and only the calls are numbered in the other, so a
call that follows a sentence is block one and call zero.

A caller on `/v1/messages` gets that dialect's events back: a message that opens,
opens a content block, fills it, closes it, and closes the message. The one
figure that cannot be where this dialect puts it is the input token count, which
the other dialect reports at the end of a stream if at all. So it is null on the
opening event and real on the closing one, rather than a zero nobody measured.
`[DONE]` is a convention rather than a guarantee, so a provider that simply
closes its socket still leaves the caller a finished message, and a stream that
breaks after its headers have gone out ends with this dialect's error event
rather than in silence.

## When a provider hangs

Version 0.1.2 had no deadline of any kind. A provider that accepted a request
and never answered held the caller and the lane for as long as the caller would
wait, and a chain behind that lane was never walked, because nothing ever
failed. Since 0.2.0 there are three budgets, in the class the provider SDKs
already use:

| Budget | Default | What it bounds |
|---|---|---|
| `VITNA_UPSTREAM_TIMEOUT_MS` | 540000 | A completion that is not streamed, from dispatch to its last byte |
| `VITNA_UPSTREAM_FIRST_BYTE_MS` | 300000 | A stream, from dispatch to its first chunk |
| `VITNA_UPSTREAM_IDLE_MS` | 300000 | A stream, between one chunk and the next |

Each is at most 2147483647, the longest a timer can hold. A longer value is
refused at boot rather than accepted, because Node fires a longer timer after
one millisecond.

**Why nine minutes, and not thirty seconds.** For a request that is not streamed,
the first byte of the response is the whole body, so a short deadline there is a
short deadline on the whole completion. Reasoning and long-context lanes take
minutes when not streamed, which is why the OpenAI and Anthropic SDKs both
default to ten. A thirty-second budget would abort a primary the provider was
still generating, and billing, and then bill again for the fallback. It is a
minute under the SDKs' ten rather than equal to it for the reason in the next
section: a budget as long as the caller's never runs out while the caller is
still there.

**Why five minutes for a stream.** The runtime waits for a stream's first
chunk, not its headers, and a reasoning model streamed direct can send no chunk
until its hidden reasoning ends. The OpenRouter latency figures the record
mirrors (read 2026-09-13) put time to first token above 30 seconds at the 90th
percentile on 15 busy endpoints, OpenAI's own among them, and the slowest near
five minutes. A budget shorter than that aborts a lane after the provider has
started billing for its reasoning, then bills a fallback. Idle is the same
figure, because a provider that opens with an empty chunk moves that silence
inside the stream. A lane that never reasons can be given shorter budgets.

**Your SDK's timeout must be longer than the runtime's budget, or a hang is
never failed over.** The SDK starts its clock before the request leaves it, and
the runtime starts its own only once the request has arrived and been routed.
When the SDK gives up first, the runtime sees a caller who left, walks no chain,
and the SDK then retries (twice, by default) against the same hung primary. With
the defaults, a default SDK's ten minutes leaves a fallback one minute to
answer. A lane that needs more, or a lane with a signed budget near ten minutes,
needs the SDK's timeout raised to cover the budget and the fallback's answer.

**What a timeout does depends on whether the caller has had anything.**

- **Before any byte reached the caller**, it is a provider that cannot serve the
  request now. It writes a `timed_out` [ledger](ledger.md) row with `latencyMs`
  at the budget, and the chain is walked exactly as it is for an unreachable
  provider. A lane with no chain answers 504. The runtime reads a stream's first
  chunk before it sends the caller anything, so a provider that opens a stream
  and then goes quiet is still in this case.
- **After bytes reached the caller**, the response is ended and the row says
  `timed_out`. It does not fail over, because a second provider's answer
  spliced after the first one's opening is not an answer. An Anthropic-dialect
  caller is sent that dialect's error event first, so it does not wait forever.

**A caller who hangs up is followed out.** When the connection closes before the
response has finished, the upstream request is aborted, a `caller_left` row is
written with `latencyMs` at the moment they left, and the chain is never walked
for them. Nobody is listening, so a fallback would be a second bill for a
prompt whose answer nobody will read.

**Both are requests the provider still bills.** An aborted HTTP request does not
refund what the provider generated before the abort reached it. The runtime's
budgets stop the second bill and the endless wait; they cannot refund the first.
So set your SDK's own timeout longer than these budgets, knowing that when it
gives up the runtime aborts and the provider keeps what it already charged, and
set these budgets knowing that a budget shorter than a lane's real completion
time pays for a failure.

**A lane may carry its own completion budget.** A signed policy may put
`upstreamTimeoutMs` on a lane, and a 0.2.0 runtime serves that lane under it in
place of `VITNA_UPSTREAM_TIMEOUT_MS`. It governs the completion budget only; the
stream budgets stay the environment's. The boot output and `/status` print the
budget each lane runs under, so a lane signed shorter can be seen to be.

It is signed into the policy's digest, the same way a seller set is. So **a
0.1.2 runtime refuses a policy that carries one**: it recomputes a digest
without the field, the digest does not match, the refusal is logged, and the
policy it was already serving stands. A policy with no budget on any lane is
byte-identical to before and verifies everywhere. 0.1.2 enforces no deadline at
all, from the environment or from a policy, so upgrade before signing a budget
into a lane.

## What it refuses

| Refusal | Status |
|---|---|
| A `/v1` request without the inbound key, where one is set | 401 |
| Where no key is set: a non-loopback `Host`, or a `/v1` request with an `Origin` or a body not sent as `application/json` | 403 |
| A model field naming nothing | 400 |
| A lane the policy does not carry, with the ones it does | 404 |
| A lane whose provider cannot be reached with the keys held | 422 |
| A lane whose every attempt is outside `VITNA_ALLOW_PROVIDERS` | 422 |
| A lane routed through OpenRouter that OpenRouter does not list | 422 |
| A model name with no lane prefix, and no key that could serve it | 422 |
| A field the Anthropic translation does not carry, such as `response_format` | 400 |
| A field this runtime does not read out of a messages request, such as `thinking` | 400 |
| A provider that did not answer within its budget, on a lane with no chain left | 504 |
| A request still under way when a restart's drain window ran out | 503 |
| A provider's own error | Whatever the provider said |

The configuration refusals are 422, and were 502 before 0.2.0. Nothing upstream
failed: the runtime was started without a way to reach that lane, which is as
true on the third attempt as on the first, and the SDKs retry a 5xx. A 422 is
answered once. A provider's own status is never rewritten, 5xx included.

A field the translation does not carry is refused by name rather than dropped: a
caller who sent `response_format`, had it dropped, and got an answer that ignored
it has been given a wrong answer that looks like a right one. Route that lane
through OpenRouter, which speaks the dialect natively.

`tools` used to be on that list, and moving it off is worth stating plainly
because the advice attached to it was wrong. A provider with a direct key takes
the direct route by default, so that refusal was the default for anybody holding
an Anthropic key, and its workaround was to force the route to OpenRouter. A
forced route says something about committed spend or data handling. Asking
somebody to say it in order to work around an incomplete translator was this
runtime reaching into a decision that is theirs.

Every one of these except the 401 and the 403 is a [ledger](ledger.md) row. A ledger holding
only successes overstates what a lane did.

## What it changes, and only where you ask

By default the runtime forwards a request as the caller wrote it, apart from the
model name a lane decides, and returns the provider's answer as it came. Four
switches change that, and each is off until you set it:

| Switch | What it changes |
|---|---|
| `VITNA_COMPACT_CONTEXT=1` | Compacts prompts before dispatch: trailing whitespace and blank runs outside code fences, identical consecutive turns, and consecutive system turns merged into one |
| `VITNA_STRUCTURAL_STOP=1` | Cuts whatever follows a JSON answer's close, for a request asking for `response_format: {"type": "json_object"}` or sending `structural_stop: true`, and ends a stream there. Until 2026-09-29 the request alone switched it on |
| `VITNA_CACHE_TTL_SECONDS` | Answers a repeat of a deterministic request from memory: [the response cache](cache.md) |
| `VITNA_REDACT_LANES`, `VITNA_REDACT_PASSTHROUGH=1` | Lifts known credential formats out of a request and puts them back into its answer: [redaction](redaction.md) |

The first two write a figure of their own on the row when they run,
`tokens_evaporated` and `tokens_early_stopped`, counted at four characters a
token, which is an estimate rather than anything a provider reported. With
neither switched on, no row carries one.

## Reloading

`SIGHUP` re-reads the policy file. A reload that cannot be verified changes
nothing: the running policy stands, the refusal is logged, and `/status` counts
it. Fail static, because adopting an unverified document routes traffic nobody
signed for, and refusing to serve turns a bad file into an outage.

A stale policy is reported and never enforced. A runtime that stopped serving
because its policy was old would convert a control-plane outage into a
data-plane one, which is the coupling this design exists to refuse.

## Restarting

`SIGTERM`, which systemd, `docker stop` and every `fly deploy` send, drains
rather than cuts. The runtime stops accepting and lets the responses already
under way finish for up to 30 seconds. Whatever is still running then is cut: a
caller still waiting for a whole answer is told 503, a stream is ended with its
dialect's error where it has one, and each writes a `drained`
[ledger](ledger.md) row before the process exits, which takes up to 2 seconds
more. A second `SIGTERM` exits at once. Before 0.2.0 a restart ended every
stream in flight mid-body.

**Whatever sends the `SIGTERM` must wait longer than that**, or it sends
`SIGKILL` mid-drain and the rows go with the process. Neither common default
does: `docker stop` waits 10 seconds and Fly waits 5. Give it 35: `kill_timeout
= 35` in `fly.toml` (the shipped recipe sets it), `docker run --stop-timeout 35`
or `docker stop -t 35`, `stop_grace_period: 35s` in Compose, and
`terminationGracePeriodSeconds: 35` in Kubernetes. systemd's default of 90
seconds already covers it.

## Status

```bash
GET /status
```

The runtime's version, whether it serves a policy (`mode` is `policy`, or
`passthrough` for [Relay](relay.md), which serves none), the policy digest, its
age, which lanes are servable, by which route and under which completion
budget, which lanes are not and why, whether each key is set, the budgets in
force, what each lane has done (including `timedOut`, `callerLeft` and
`cacheHits`), what the ledger has written, and what the
[response cache](cache.md) and [redaction](redaction.md) have done, as counts.

With `VITNA_INBOUND_KEY` set, that page needs the key. Without the header it is
`{ "ok": true, "version": "0.2.0" }`, which is also what `GET /healthz` always
answers, so a health check needs no credentials and learns nothing else.

Keys appear as `set` or `unset`. Never a value, never a prefix of one.

A lane that cannot be served does not stop the others starting. A missing key
for one provider must not take down the lanes of another, so the runtime starts,
names what it cannot serve, and refuses those requests individually.
