> Source: https://vitna.ai/docs/runtime/cache > Raw Markdown: https://vitna.ai/docs/runtime/cache.md > Corpus index: https://vitna.ai/llms.txt # The response cache A repeat of a deterministic request, answered from memory rather than by a provider. Off unless you turn it on, and never a saving. ```bash VITNA_CACHE_TTL_SECONDS=300 vitna-runtime vitna-relay --cache 300 ``` ## What it answers Only a request that asked for the most likely answer, once, and whole: - `temperature` set to `0`. An absent temperature is the provider's default, which samples, so a request that did not say `0` is never answered from memory. - One choice: `n` absent or `1`. - Not streamed. And only one the runtime reads as a chat completion. An Anthropic request that names a model is [passed to Anthropic as it was written](serving.md#the-other-dialect), and the cache leaves it alone as well. Zero is not a promise that a provider answers the same way twice. What the cache promises is narrower: an answer it gives is the answer a provider gave to exactly this request, within the last `VITNA_CACHE_TTL_SECONDS`. ## Exactly this request The key is the whole body the runtime would forward, in a canonical form in which key order does not matter and everything else does: the messages to the last character, the tool calls and their ids, `max_tokens`, `stop`, `tool_choice`, every field. For a lane it also holds the policy digest, the lane, and the caller's `x-vitna-cohort`, since those decide which attempt serves the request. So a reload that changes what a lane serves never answers with what the previous decision put there, and two callers on different sides of a traffic share never answer each other. ## A hit The answer is the one stored, with `x-vitna-cache: hit` on the response. Its [ledger](ledger.md) row says what happened and nothing more: | Field | On a hit | |---|---| | `serve`, `via` | `cache` | | `cache_hit` | `true` | | `outcome` | `served`, because the caller got an answer | | `units` | `0`, because no provider served it | | `inputTokens`, `outputTokens` and the cache parts | null, because nobody was asked anything | | `sku`, `decision`, `provider`, `model` | the serve whose answer it repeats, so the answer still resolves to the decision that authorized it | `vitna-runtime-push` and `vitna-runtime-profile` count hits apart from their figures and say how many, so a lane answered from memory neither reads as a sku that failed nor inflates what the sku did. `/status` counts hits, misses and entries. Nothing anywhere adds up what a hit would have cost: a price needs the date it applied on, and the runtime counts nothing in money. ## Asking for a fresh answer A caller's `cache-control` header is honoured: `no-cache` asks the provider even where an answer is stored, and replaces it; `no-store` keeps this request's answer out of memory. ## What it holds At most 1024 answers and 64 MB, the least recently used going first, in the serving process's memory. Nothing is written to disk, and a restart starts it empty. An answer larger than the whole allowance is not kept. Answers are kept after [redaction](redaction.md) has put a request's secrets back into them, so a cached answer may hold a secret the caller sent. It stays in the same process that already held it.