Grounding check
platform/search/grounding.mjs decides, segment by segment, whether an answer a model wrote from Vitna Search evidence is supported by the verbatim passages it cites. It is deterministic and lexical: it uses no model and no network, and it checks that what a sentence states occurs in the cited passages, not what the sentence means.
Where it sits
Search supplies the evidence. Source brief research returns verbatim passages, each with a citation id, a source, and code-point offsets into the stored text version read for that source. The answer is written outside Search, by a model the caller runs, from those passages only, citing them with inline markers such as [c1]. The verifier then reads the evidence and the answer together and says which segments the evidence supports, and returns the supported text alone.
It runs where the answer is written, in the caller's process, reads the two values it is handed, and sends nothing anywhere. The file imports nothing and uses no regular-expression lookbehind, so it can be copied whole into code for Node, Deno, Bun, Workers or a browser; it is not yet published in the JavaScript or Python SDK.
import { verifyAnswer } from "./grounding.mjs";
const evidence = await response.json(); // the /v1/research response, as returned
const result = verifyAnswer(evidence, answer, { minOverlap: 0.45 });minOverlap is optional and defaults to GROUNDING_MIN_OVERLAP, the measured 0.45. The module also exports parseCitedAnswer(text), which returns the answer's segments as {text, citationIds, kind} with markers removed, and the INSUFFICIENT_EVIDENCE sentinel.
supported means the cited passages hold the segment's figures, quotations and names and most of its words. It does not mean the segment is true: passages are publisher text, and the checks below cannot see a reversed meaning.
Evidence
The verifier reads sources, citations, and the passage texts quoted in overview and sections[].passages. A top-level passages list, or a text on the citation itself, is read too. Several research responses merged into one pack work the same way, provided their source and citation ids are renumbered to stay unique. A citation is usable only when its id is unique, offsetUnit is unicode_code_points, start and end are integers with start below end, sourceId names a listed source, exactly one passage text is quoted for it, and that text is end - start code points long. When a source carries its text, the passage must also be that slice of it.
A citation that fails is listed in evidence.invalid with its reason: duplicate_id, offset_unit, offsets, unknown_source, missing_text, conflicting_text, too_long, length_mismatch or source_text_mismatch. A segment citing it is refused with invalid_citation instead of being checked against it.
Answer
The answer is Markdown text with inline markers, or a list of segments, [{ text, citationIds }].
## Opening
It opened to pedestrians on May 27, 1937 [c1]. Cars crossed on May 29, 1937 [c1].Text is parsed into segments: each heading (a # line, or a line that is only bold text), and each sentence of a paragraph or list item. Sentences end at ., ! or ? followed by a capital, a digit or a quotation mark, except after common abbreviations such as Dr., e.g. and U.S., and after a single initial. Markers just after a sentence's full stop belong to that sentence, and a line holding only markers belongs to the sentence before it. A bracket is a marker when each comma-separated part is an id in the evidence or has the shape of one, letters then digits or digits alone, so [1] is refused as an unknown id rather than read as the figure 1. A link's bracketed text is not a marker. Fences and horizontal rules are markup; the text inside a fence is read as a paragraph.
A segment list is used as given, with any inline markers in a segment's text added to its ids. A segment whose text opens with #, or whose kind is heading, is a heading; one opening with a list marker is a list item.
When the passages do not answer the question, the model should reply with INSUFFICIENT_EVIDENCE alone. An answer that is only that word, allowing surrounding whitespace, quotes, backticks or a full stop, has no segments: coverage.status is insufficient and coverage.abstained is true. Beside other text the word is an uncited sentence and is refused.
Output
{
"segments": [
{"index": 0, "kind": "heading", "text": "Opening", "citationIds": [], "claim": false, "supported": true, "reasons": [], "details": [], "overlap": null},
{"index": 1, "kind": "sentence", "text": "It opened to pedestrians on May 27, 1937.", "citationIds": ["c1"], "claim": true, "supported": true, "reasons": [], "details": [], "overlap": 1},
{"index": 2, "kind": "sentence", "text": "Cars crossed on May 29, 1937.", "citationIds": ["c1"], "claim": true, "supported": false,
"reasons": ["unsupported_number", "low_overlap"], "details": [{"reason": "unsupported_number", "text": "May 29, 1937"}, {"reason": "low_overlap", "text": "cars crossed"}], "overlap": 0}
],
"supportedText": "## Opening\n\nIt opened to pedestrians on May 27, 1937.",
"coverage": {"status": "partial", "segments": 3, "supported": 2, "unsupported": 1, "claims": 2, "supportedClaims": 1, "abstained": false},
"evidence": {"citations": 1, "usable": 1, "invalid": []},
"minOverlap": 0.45
}| Field | Meaning |
|---|---|
segments[].text | The segment with its markers, list marker and emphasis removed |
segments[].kind | heading, lead_in, connective, closing, item or sentence |
segments[].claim | false for the four structural kinds, which need no citation |
segments[].supported | Every check passed |
segments[].reasons | Reason codes, in the order of the table below |
segments[].details | Each failing token as {reason, text}; for low_overlap, the content words no cited passage holds |
segments[].overlap | Share of the segment's content words found in its cited passages, or null |
supportedText | The supported segments with their headings and list markers, citation markers removed |
coverage.status | matched, partial or insufficient, as research uses them |
coverage.supported / unsupported | Segments supported and refused, structural ones included |
coverage.claims / supportedClaims | Segments that are claims, and those supported |
coverage.abstained | The answer was only INSUFFICIENT_EVIDENCE |
coverage.status is matched when at least one claim is supported and no segment is refused, partial when some claim is supported and some segment is refused, and insufficient when no claim is supported. A refused heading or lead-in counts as refused. In supportedText a heading is kept only over a supported claim, a lead-in only before one, and a connective or closing sentence only when some claim survived. The segments keep their citationIds, so a reader can show the supported text with its sources.
Checks
A claim is any segment that is not structural. Each claim is checked against the usable passages it cites, and every failure is reported, not only the first.
| Reason | When |
|---|---|
missing_citation | The claim cites nothing |
unknown_citation | It cites an id the evidence does not hold, such as an invented c9 or a source id like s3 |
invalid_citation | It cites a citation that failed the evidence checks |
unsupported_number | A number, percentage, money amount, year or date is in no cited passage |
unsupported_quote | A quoted string is in no cited passage |
unsupported_name | A capitalised name is in no cited passage |
low_overlap | Fewer than 0.45 of its content words occur in the cited passages |
One unknown or invalid id refuses a claim even beside a good one. The content checks still run against the usable ids, so every problem is listed.
Numbers are compared as values. 1,200,000, 1.2 million and $1.2m are the same value; a currency symbol or word must match; a percentage must meet a percentage; a bare number may meet a passage number of any kind. Spelled numbers count (eleven, twenty-five, a thousand), and a lone one counts only as a quantity, not in one of or no one. A figure hedged with about, roughly, nearly or the like may round the passage's figure to its own precision, so about 40% meets 38.7% but about 99% does not meet 97%. An unhedged figure must match exactly.
Dates are read in their written forms (May 27, 1937, 27 May 1937, 2022-08-18, August 2022, May 27) and compared as dates. A date is supported by one at least as specific, so May 1937 meets May 27, 1937, and a passage's 12 March meets 12 March 2021 when the passage holds 2021 elsewhere.
Quotations in straight or curly double quotes, guillemets, and single quotes that are not apostrophes must occur as a word sequence in one cited passage, ignoring case, punctuation and diacritics.
Names are runs of capitalised words, joined across of, de, van and the like, matched as a word sequence in one cited passage, ignoring case and inflection. Month and weekday names are left to the date check. A sentence's first word joins a name only when the evidence writes it capitalised mid-sentence, or when it starts a longer run and is not an ordinary word of the cited passages, so Brooklyn Bridge is checked whole while Typically is not a name. Headings are title case, so their capitals are not read as names.
Content words are the segment's words less function words, discourse words such as overall and according, numbers, and month names, after a light suffix stripper on both sides (spans meets spanning, released meets releases). Diacritics and letters such as ł fold, so Sklodowska meets Skłodowska.
Structural text
Four kinds of segment make no claim of their own and need no citation, each only while it is 20 words or shorter.
| Kind | What it is |
|---|---|
heading | A Markdown heading, or a bold-only line |
lead_in | An uncited sentence ending in a colon, such as Bakers watch a few signals: |
connective | An uncited segment with no content word, figure or quotation, such as Hope this helps! |
closing | The answer's last segment, uncited, that passes every check, overlap included, against the passages the answer's supported claims cited |
Structural text still answers for what it states. A number, quotation or name in a heading, lead-in or connective must occur in the passages it cites or, citing none, somewhere in the evidence; so # Opened in 1938 is refused when no passage says 1938. A closing sentence that fails any check is an uncited claim and is refused with missing_citation. A list item is a claim like any sentence.
Measured precision and recall
npm run search:grounding-evaluateThe command verifies twelve labelled answers in platform/search/grounding-fixture.mjs and writes .search-data/reports/grounding.json (or --output path) with the threshold sweep, the results per split and per kind, and every misjudged segment. Each answer is written out as the Markdown a model would return and verified whole, so the parser is measured with the checks; a pack that does not parse into one segment per labelled row fails the run.
The fixture holds 230 segments over twelve evidence packs shaped like research responses: 115 supported and 115 adversarial. Six packs choose the threshold and six held-out packs only measure it. The threshold is the lowest with the best F0.5 of predicting supported on the development packs, precision weighted over recall because a passed unsupported sentence costs a reader more than a refused one. On the development packs F0.5 is 0.866 at both 0.45 and 0.5. Recall starts to fall at 0.55, and precision rises again only from 0.9, where the overlap refuses some named limits along with a third of the supported segments. The module's 0.45 is that choice, and a test holds the two equal.
| Packs | Segments | Precision | Recall |
|---|---|---|---|
| Development | 114 | 0.838 | 1.000 |
| Held out | 116 | 0.871 | 0.931 |
| All packs | 230 | 0.854 | 0.965 |
| All packs, named limits left out | 211 | 1.000 | 0.965 |
Precision is held below 1 by the named limits below, which the adversarial half includes on purpose. Every other adversarial segment is refused.
| Kind | Label | Segments | Judged as labelled |
|---|---|---|---|
verbatim | Supported | 31 | 31 |
paraphrase | Supported | 34 | 33 |
synthesis | Supported | 6 | 6 |
heading | Supported | 12 | 12 |
lead_in | Supported | 7 | 7 |
list_item | Supported | 13 | 13 |
closing | Supported | 12 | 9 |
changed_number | Adversarial | 30 | 30 |
changed_date | Adversarial | 8 | 8 |
swapped_name | Adversarial | 12 | 12 |
uncited | Adversarial | 12 | 12 |
invented_id | Adversarial | 12 | 12 |
merged | Adversarial | 12 | 12 |
wrong_quote | Adversarial | 7 | 7 |
embellished | Adversarial | 3 | 3 |
negation | Named limit | 6 | 0 |
comparison | Named limit | 5 | 0 |
role_swap | Named limit | 5 | 0 |
substitution | Named limit | 3 | 0 |
The four supported segments refused are all held out: three closing sentences that add an evaluation the evidence does not make (the budget grows modestly, two doses give strong protection, pressure, dose and time decide the shot), and one paraphrase sharing two of five content words with its passage (Its net loss shrank to $6.1 million, compared with $9.8 million a year before).
The fixture is synthetic and was written by one author who knew the checks. The two Calder County and Northwind Robotics packs are invented; the rest follow well-known public facts, but each pack asserts only what its own passages say. It measures the checks on the failure kinds named here, not a model's real error rate, and a model's errors will include subtler ones.
Limits
A lexical check confirms that tokens occur, not what a sentence does with them. These pass, and the fixture counts them as passed unsupported segments:
- Negation.
The bridge is not painted International Orangekeeps every word of its passage;not,noandneverare function words to the overlap, and no other check reads polarity. - Reversed comparison or change.
fallsforrises,coolerforwarmer,widenedfornarrowed: one changed word in a sentence otherwise drawn from the passage leaves the overlap high. - Swapped roles. Two entities or two figures of the same passage exchanged, as in
Aldrin became the first person to step onto the lunar surface, or$15.9 million, up from $18.2 millionfor the reverse. That includes a figure moved to another unit or quantity, such as21.5 poundswhere the passage has47.5 pounds (21.5 kg): units are not compared. - Substitution. One ordinary word replaced in a long sentence, such as
stromaforthylakoid membranesorsecondforfirst. Spelled ordinals are words, not numbers.
Other edges, measured or known:
- A count in a lead-in, such as
Two changes matter most:, must occur in the evidence, and can be met by coincidence. Both of the fixture's passed that way, one onThe two astronautsand the other on the day ofOctober 2. - A paraphrase that shares too few words with its passage is refused, and so is a closing sentence that evaluates rather than restates.
- Only hedged figures round.
1.2 millionwithoutaboutmust match exactly. - Stopwords, number words, month names and the suffix stripper are English.
- The verifier trusts a passage as far as its length, and its slice when the source text is supplied. It fetches nothing, so it cannot tell whether the passage is still what the page says; a later crawl can change the offsets.
