Get started
DocumentationConcepts

Source

Source

One fetchable document, and the adapter that knows how to read it.

Sources are registry rows in witness.source.

Fields

FieldMeaning
idStable source id
providerIdWho publishes it
urlWhat the collector fetches
publicUrlThe page a citation should point a human at
adapterThe parser that reads this document
enabledWhether the collector reads it
cadenceMinutesHow often, default 60
robotsStatusWhat robots.txt said
notesFree text

Why url and publicUrl are two columns

What a machine should read is not always what a person should read. Anthropic's pricing parses cleanly from the documentation's Markdown mirror; Google's needs a locale-pinned URL to answer consistently.

Citing those would send a reader to a mirror or to a URL carrying a locale parameter they did not choose. So the tape cites publicUrl and the collector fetches url, and a citation never points at scaffolding.

Provider domains are the inclusion rule

witness.provider carries a domains array, and it does real work. A curated market event is refused unless its source URL sits on one of the provider's own registrable domains, subdomains included.

That turns "is this the provider's own announcement?" from a judgement call made per entry into a constraint the database enforces. A press summary, a news article or a well-informed blog post cannot become a curated event, however accurate it is.

Health

bash
curl https://vitna.ai/api/v1/sources/health

Public, deliberately. A source that has stopped answering is part of the record: a tape that silently stopped updating because a page moved looks exactly like a market that stopped moving.

The endpoint flags a source stale past its cadence and failing at three or more consecutive failures. It answers 503 when the record cannot be read at all, rather than reporting green from a committed file, which would be the one lie this system exists to prevent.

Fetch outcomes

Every read writes one witness.fetch row with a final status, inserted once after both the fetch and the parse have been decided, so nothing about a fetch is ever rewritten:

StatusMeaning
okFetched and parsed. Has a contentSha256.
fetch_failedCould not be retrieved. Has an error.
parse_failedRetrieved, but the adapter could not read it. Has an error.
robots_blockedrobots.txt disallowed the read.

Two database constraints hold the shape: an ok row must have evidence, and a non-ok row must have a reason.