Get started

The standalone scripts

Source

The mirrors that write app/data/, the reconcilers that run behind them, and the rest of what is run with node rather than through npm.

ScriptWhat it does
export-snapshot.mjsMirror the live tape into app/data/*.json
export-catalog.mjsMirror the public model list into app/data/catalog.json
export-speed.mjsMirror OpenRouter's measured throughput and latency into app/data/speed.json
export-endpoints.mjsMirror what every seller charges, and what it declares about its route, into app/data/endpoints.json, and one uptime reading a day per seller into app/data/uptime.json
export-images.mjsMirror what each image model costs, per endpoint and in the unit each price is published in, into app/data/image-models.json
export-weights.mjsMirror what each open-weight listing's Hugging Face repository says about itself into app/data/weights.json
export-eci.mjsMirror Epoch AI's Capabilities Index into app/data/eci.json
export-usage.mjsMirror OpenRouter's usage datasets into app/data/usage.json, with a key
reconcile-catalog.mjsMake the speed, seller, uptime, weights and sku-map documents agree with a fresh catalog
reconcile-registry.mjsRecord which registered skus the boards have stopped pricing
triage-candidates.mjsSort the discovery queue into promote, reseller, tiered and unlocatable
shoot.mjsCapture every route as full-page and viewport images
edit-fixture-price.mjsAn edited copy of an archived document, for scenarios only

The mirrors

bash
node scripts/export-snapshot.mjs https://vitna.ai
node scripts/export-catalog.mjs --dry-run
node scripts/export-speed.mjs --limit 20

All three write into app/data/, which is what the site renders when it has no database. That keeps two logs at two resolutions: git is the daily one, with a commit per day and a readable diff, and the database is the hourly one.

export-catalog.mjs is key-free and writes only when a row moved, so a run that changes nothing produces no commit.

The two reconcilers

bash
node scripts/reconcile-catalog.mjs --check
node scripts/reconcile-registry.mjs https://vitna.ai --check

Each runs immediately after the mirror it belongs to, in the same job, so that one commit carries a document and everything that document implies. Neither decides anything; both carry a decision already made upstream into the files that restate it. --check reports and writes nothing.

They exist because a withdrawal used to stop a mirror dead. A product leaves a market, the fresh document correctly stops carrying it, and a hand-written file somewhere still names it, so a test refuses the pair and the only file that could resolve it is the one being refused. Between 2026-08-25 and 2026-09-08 both the catalog and the snapshot mirrors failed on that shape every single day, and the site could not accept a new model for a fortnight.

reconcile-catalog.mjs reads no network. It deletes the speed and seller rows for listings the fresh catalog no longer carries, recounting each document so it still clears the floors a freshly read one has to, and records which of the sku map's ids have been withdrawn in platform/catalog/delisted.mjs, which the map is read through.

reconcile-registry.mjs reads the boards and /api/v1/sources/health, and writes platform/witness/withdrawn-skus.mjs. Nothing is deleted from the register: an adapter replaying an archived document, a deprecation event naming its sku, and a sku's price history all need the product to stay registered after it stops being sold, so the withdrawal is a list beside the register rather than a cut into it.

The health read is the part worth understanding. A board absence means "no current price", and a source whose parser is failing produces no current price for anything it quotes, so under a broken adapter an absence says nothing about the market at all. On the day this was written, eleven of seventeen absences were real withdrawals attested by sources that had read their pages that morning; the other six were Novita products still on sale behind a parser that had failed 208 times running. Only a healthy source can put a row in withdrawnSkus. The rest go to unattestedSkus, which the boards are compared against too, so a broken source no longer stalls the mirror, and the file says which of the two happened to every row.

export-speed.mjs is key-free too, and reads differently: OpenRouter's API defines latency_last_30m and throughput_last_30m and returns null for both on every endpoint, so the figures are read from each model's own page instead, one page per model, three at a time. It asks robots.txt once before the run and sends the same honest agent as the collector. The list of pages is the committed catalog, so it can never read a page the catalog does not already carry.

Unlike the catalog, this one changes almost every run, because the market's speed does. A page that will not answer is a recorded gap in meta.failures, never a zero, and a run that reached fewer pages and measured fewer models than the committed document refuses to write.

--limit and --only make a smoke test cheap. Neither writes the document: a partial read is not a picture of the market.

export-images.mjs is the price list for images. It reads OpenRouter's image models list, then each listed model's endpoints document, key-free, three at a time after asking robots.txt once. It does not start from the catalog, because most image models are not in it. Every price is kept in the unit it is published in, per image, per megapixel, per token or per request, beside what it bills and the condition it holds under, and nothing converts one unit into another: a price per token is a price per image only after a run says how many tokens the image took. A listed model nobody serves is named in meta.unserved, a served model that publishes no price in meta.unpriced, and a document that did not answer in meta.failures, because those are three different answers to whether a picker can show a price. It claims no licence: OpenRouter's CC BY 4.0 grant covers its keyed Data API and, by its own terms, nothing published outside it.

export-usage.mjs is the one mirror that needs a key. OpenRouter publishes the figures behind its rankings page, tokens per model per UTC day and tokens per app over the window, at two dataset endpoints that answer 401 without a valid OpenRouter key. The data is CC BY 4.0 and the citation names OpenRouter and the as_of instant each response carries. The key is read from OPENROUTER_API_KEY, put in one request header and nowhere else, and the workflow passes the secret to that one step. Two requests a run, against a limit of 500 a day.

bash
OPENROUTER_API_KEY=... node scripts/export-usage.mjs --dry-run
node scripts/export-usage.mjs --from <dir>

--from parses rankings-daily.json and app-rankings.json saved in a directory, dated by the date response header saved beside each as <name>.headers, so the document can be rebuilt from a saved read without a key. A read with a new dataset version, a row after the day's other row, a count past the exact range of a number, or fewer than 20 days, 30 models or 20 apps is refused and the committed document stands.

export-weights.mjs is the one mirror that reads a host other than OpenRouter. The catalog names a Hugging Face repository for every listing whose weights are published, and the repository's own metadata carries four facts the list does not: the licence the uploader tagged it with, the parameter count its safetensors declare, the model class its config names, and whether the download is gated behind accepting the maker's terms. One request per repository, key-free, three at a time, after asking robots.txt once. The document is keyed on the repository rather than the listing, because a base listing and its variants point at one set of weights.

What it deliberately does not read is downloads and likes. Those are adoption, which the catalog pages say they are not, and a figure that moves every hour would make every run write. A licence is printed as the uploader tagged it, never as verified, and a repository that does not answer is a recorded gap in meta.failures, never a model with no licence.

export-eci.mjs carries one published index rather than a table of scores: Epoch AI's Capabilities Index, every model it places with the interval around its figure. It runs in the board mirror's job, behind the licence watch, because it is covered by the same licence row as Epoch's board, and it refuses to fetch anything if that row stops permitting republication. Like the boards, a publisher that does not answer leaves the committed document standing and the run green; a document that answers and no longer parses stops the run.

It reads two files and republishes one. The second, the results behind each figure, mixes Epoch's own runs with results collected from other publishers, whose licences still apply, so only the count per model is kept.

--pairs prints which catalog listing each model is. The pairing takes a reviewed decision, an equal display name, or an exact slug through the same collision-refusing index the board names use, and it refuses a listing that was on sale more than a fortnight before the model was released, which is what pairing a later snapshot onto an earlier listing looks like.

edit-fixture-price.mjs is for scenarios only

It produces a deliberately edited copy of an archived document, so a test can exercise the change detection path against a known before and after.

It exists so that no test ever needs to invent a document. The edited copy is labelled as one and never enters the evidence archive.