Groundedness in AI: What It Measures and How It Is Scored

What Is Groundedness in AI?

Groundedness in AI is the share of an answer's claims that the sources supplied with the request support. Those sources, often called the context, are whatever the model was given to answer from: retrieved passages, a document to summarize or the response of a tool. Groundedness asks whether the model stayed within that material and leaves the truth of the material to other checks.

Microsoft's GroundednessEvaluator reference (updated 29 September 2026) states the boundary: "Even if the responses from LLM are factually correct, they'll be considered ungrounded if they can't be verified against the provided sources (such as your input source or your database)." The same evaluator scores groundedness from 1 to 5, with 5 the most grounded.

Outside AI, the word describes a calm, practical person. The sense used here is the evaluation metric for retrieval-augmented generation (RAG), summarization and AI agents, and two answers built on the same source show the difference:

• Grounded - the source says "Requests to the Orders API are limited to 100 per minute for each API key", and the answer says "Each key can make 100 requests a minute." Every claim traces back to the source.
• Ungrounded - the answer adds "and unused requests roll over to the next minute". Nothing in the source says so, which makes the answer ungrounded whether or not the API behaves that way.

How Is a Groundedness Score Calculated?

Every groundedness check decides how to split the answer, what counts as support and how to combine the results, and each choice moves the score.

• Split the answer into claims - the Vertex AI check grounding documentation (updated 13 October 2026) says that "a sentence is considered a single claim". TruLens, an open-source evaluation library, splits the response into sentences with a tokenizer and removes trivial statements "so as to not dilute the evaluation", according to its documentation. Claim-level methods go further: FActScore (EMNLP 2023) breaks a generation into atomic facts and reports the share of them that a reliable knowledge source supports.
• Decide whether the context supports each claim - a natural language inference (NLI) model or an LLM judge reads each claim beside the context and labels it supported or not. The NAACL 2024 study above used TRUE, a T5-11B model trained on NLI datasets, and its annotators agreed with that model in 82% of the cases they checked. The guide to LLM-as-a-judge covers how judge models are made reliable and which biases to guard against.
• Combine the results into a score - the plain version reports the share of claims supported, and a strict rule fails the whole answer when any claim is unsupported. Judge-based libraries average ratings instead: TruLens rates each statement from 0 to 3 by default and averages the normalized ratings, and the Ragas Response Groundedness metric takes two judge ratings on a 0 to 2 scale and averages them.

groundedness = supported claims / all claims

strict rule: the answer is grounded only
             when no claim is unsupported

Support means that the context implies the claim. A claim the context never mentions is unsupported even when it is true, and a sentence that mixes a supported fact with an unsupported one needs a rule of its own. Google's documentation gives the case of "Google was founded by Larry Page and Sergey Brin in 1975": the founders are right and the year is wrong, so the whole claim is ungrounded.

For chatbots, the guide to chatbot metrics pairs this per-claim score with a per-response hallucination rate and gives both formulas.

How Do You Score One Answer for Groundedness by Hand?

The example below is sample data: an imaginary Orders API, two chunks its docs assistant retrieved, and a six-sentence answer. The labels were set by hand, and every rule after them reads the same labels, so any difference in the scores comes from the rule.

SAMPLE DATA: an imaginary API and its docs assistant

QUESTION
What happens if I go over the Orders API rate limit?

CONTEXT (two retrieved chunks)
[c1] Requests to the Orders API are limited to 100 per
     minute for each API key. A request over the limit
     returns HTTP 429 with a Retry-After header, in
     seconds.
[c2] The limit resets at the start of each calendar
     minute (UTC). Enterprise plans can ask support
     for a higher limit.

ANSWER
S0 Here is what the Orders API documentation says.
S1 Each API key can send 100 requests per minute.
S2 A request over the limit gets HTTP 429 with a
   Retry-After header in seconds.
S3 429 is the standard HTTP status code for Too Many
   Requests.
S4 The limit resets 60 seconds after your first
   request.
S5 Enterprise plans can ask support to raise it to
   1,000 requests per minute.

Each sentence gets a label against the two chunks:

• S0 - no claim. It is a lead-in, and Google's API marks a line such as "Here is what I found." as needing no grounding check.
• S1 - supported: c1 gives the limit of 100 requests per minute for each key.
• S2 - two claims, both supported by c1: an over-limit request returns HTTP 429, and the Retry-After header is in seconds.
• S3 - not in the context. The claim is true, and no chunk says it.
• S4 - contradicts c2, which resets the limit at the start of each calendar minute.
• S5 - two claims. Asking support for a higher limit is supported by c2, and the figure of 1,000 requests per minute appears in no chunk.

The script below applies each counting rule to those labels, and Node.js v25.5.0 ran it on 16 October 2026. Its 0-3 and 0-2 rows follow the scales that TruLens and Ragas document, with ratings set by hand to match the labels; a judge model could rate the same sentences differently.

// groundedness-rules.mjs
// Sample data: one answer, claim labels set by hand.
// No model call and no network call.

// [sentence, claim, label]
const claims = [
  ["S1", "100 requests per minute per key", "supported"],
  ["S2", "over the limit returns HTTP 429", "supported"],
  ["S2", "Retry-After header in seconds", "supported"],
  ["S3", "429 means Too Many Requests", "not in context"],
  ["S4", "resets 60 s after first call", "contradicts"],
  ["S5", "Enterprise can ask for more", "supported"],
  ["S5", "the higher limit is 1,000", "not in context"],
];

// Judge-style ratings, also set by hand
const rating0to3 = { S1: 3, S2: 3, S3: 0, S4: 0, S5: 2 };
const twoRatings0to2 = [1, 1];

const ok = (c) => c[2] === "supported";
const ids = [...new Set(claims.map((c) => c[0]))];
const inS = (s) => claims.filter((c) => c[0] === s);
const whole = ids.filter((s) => inS(s).every(ok));
const part = ids.filter((s) => inS(s).some(ok));
const missing = claims.filter((c) => !ok(c)).length;
const sum = (xs) => xs.reduce((a, b) => a + b);
const avg = (xs) => sum(xs) / xs.length;

const rows = [
  ["Claims supported / all claims",
    claims.filter(ok).length / claims.length],
  ["Sentences, partial = unsupported",
    whole.length / ids.length],
  ["Sentences, partial = supported",
    part.length / ids.length],
  ["0-3 per sentence, averaged",
    avg(Object.values(rating0to3).map((r) => r / 3))],
  ["Two 0-2 ratings, averaged",
    avg(twoRatings0to2.map((r) => r / 2))],
];

const GATE = 0.5; // an example gate, not a recommendation
const col = (s, n) => String(s).padEnd(n);
console.log("SAMPLE DATA: " + ids.length + " sentences"
  + " with claims, " + claims.length + " claims");
console.log(col("Rule", 34) + col("Score", 8)
  + "Gate 0.50");
for (const [rule, score] of rows) {
  const verdict = score >= GATE ? "pass" : "FAIL";
  console.log(col(rule, 34) + col(score.toFixed(2), 8)
    + verdict);
}
const strict = missing
  ? "FAIL (" + missing + " claims)" : "pass";
console.log(col("Any unsupported claim fails", 34)
  + col("n/a", 8) + strict);

node groundedness-rules.mjs

SAMPLE DATA: 5 sentences with claims, 7 claims
Rule                              Score   Gate 0.50
Claims supported / all claims     0.57    pass
Sentences, partial = unsupported  0.40    FAIL
Sentences, partial = supported    0.60    pass
0-3 per sentence, averaged        0.53    pass
Two 0-2 ratings, averaged         0.50    pass
Any unsupported claim fails       n/a     FAIL (3 claims)

• The labels never changed, yet the scores range from 0.40 to 0.60 and the strict rule fails the answer. At the example gate of 0.50, four rules pass the answer and two fail it, so record the rule beside every threshold you set.
• S5 alone moves the sentence rules from 0.40 to 0.60: one invented number inside an otherwise supported sentence. Under the partial-support rule Google documents, that sentence counts as ungrounded.
• S3 is true: RFC 6585 assigns status code 429, Too Many Requests, to rate limiting. Every rule still counts S3 against the answer, because no chunk says it.
• S4 and the 1,000 in S5 are the claims a developer would act on, and only S4 contradicts the context. A check that looks only for contradictions passes the invented 1,000.

How Do Azure, Vertex AI and Bedrock Score Groundedness?

Each cloud documents a different rule, and the evaluation libraries TruLens and Ragas add their own. The table follows each tool's documentation as read on 16 October 2026, including Microsoft's page on groundedness detection in Azure AI Content Safety (updated 5 June 2026) and Amazon's guide to the Bedrock contextual grounding check.

A 4 from Microsoft's evaluator is a judge's rating on an ordinal scale, so reading it as 80% of claims supported adds a meaning the scale does not have. Bedrock's confidence score and the Vertex AI support score each carry their own definition. Before comparing tools, score the same hand-labeled answers with each one and compare the results with your labels.

Groundedness vs Faithfulness, Answer Relevance and Hallucination

These terms overlap, and evaluation libraries use them inconsistently. Each answers a different question, which decides the failures it can catch:

Groundedness and faithfulness often name the same claim check, so compare the formula a library documents rather than the metric name. Where a check also needs the truth, the comparison of factuality vs faithfulness metrics for RAG shows how a stale chunk pulls the two kinds of score apart.

Answer relevance catches an answer built entirely from the context that does not answer the question. Amazon's documentation gives the case: with a source naming the capitals of the UK and Japan and a question about Japan, the reply "The capital of UK is London" is grounded and irrelevant. In the FACTS Grounding benchmark paper (January 2025), responses that do not fulfill the user request are disqualified before grounding is judged, which lowered final factuality scores by 1% to 5%.

TruLens groups groundedness with context relevance and answer relevance as the RAG triad, which checks retrieval, support and usefulness on the same answer; the roundup of RAG evaluation tools covers the libraries that score it.

Hallucination is the widest of these terms. A groundedness check catches the hallucinations that go beyond or against the supplied sources and has nothing to compare when no sources were supplied. The AI hallucination explainer sorts the types, and LLM hallucination detection walks through methods that need no source, such as semantic entropy.

Groundedness vs LLM Grounding: The Technique and the Measurement

Google Cloud's grounding overview (updated 15 October 2026) defines the technique: "In generative AI, grounding is the ability to connect model output to verifiable sources of information." LLM grounding happens before the answer, when you choose what the model sees, and groundedness is measured after it, on each answer, against what the model saw.

The two words meet in the APIs: Microsoft's groundedness detection takes a "groundingSources" field and Amazon's check takes a "grounding source", the material a groundedness score is computed against. Each grounding technique calls for its own check:

Fine-tuning is a different technique. It changes the model's weights, so a fine-tuned model answers with no source passage in the request and leaves nothing to score groundedness against. In Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs, Ovadia and colleagues report that RAG "consistently outperforms" unsupervised fine-tuning, "both for existing knowledge encountered during training and entirely new knowledge", and that LLMs "struggle to learn new factual information through unsupervised fine-tuning."

What Is a Good Groundedness Score for a Release Gate?

No groundedness threshold carries over between tools, and the defaults that exist are settings of one tool. Microsoft's GroundednessEvaluator uses a default threshold of 3 on its 1 to 5 scale, and Amazon Bedrock has you choose a threshold between 0 and 0.99.

Bedrock's threshold is a runtime filter: in Amazon's words, responses that score below it "will be detected as hallucinations and blocked in your application". A guardrail filter decides what one user sees in production. A release gate decides whether a new prompt, model or index ships, and it needs a fixed set of labeled questions, a recorded baseline and a rule written before the run.

Set the gate from your own data:

• A strict rule where one wrong value costs money - for prices, limits, dates and policy terms, use the worked example's strict rule and fail the answer when any claim is unsupported.
• A baseline everywhere else - score a fixed set of labeled questions with each release candidate, and block the release on a drop from the last accepted run, measured with the same tool and settings.
• A judge from another model family - in the FACTS Grounding paper, judge models rated their own responses higher than other models' responses, by a mean of 3.23%, and the benchmark averages three judges, from Google, OpenAI and Anthropic, to reduce that bias.

Conversational agents need the same gate before release. Agent Testing from TestMu AI puts simulated personas in front of chat, voice, phone, video and image agents, and on chat and voice it scores Hallucination Detection, one of nine quality metrics, which flags information an agent invents that its knowledge base or context does not support. For chat agents, Data Validation compares each fact the agent states, a price or an order status for example, with your system of record and returns Pass, Fail or Cannot verify per fact.

What Can a Groundedness Score Not Catch?

A groundedness score answers one question about one answer, and these failures fall outside it:

• A wrong or stale source - the check trusts the context, so an answer copied accurately from an outdated chunk scores as grounded. Microsoft's Content Safety guidance tells you to "ensure that your grounding sources are accurate and up-to-date".
• A correct answer from outside the sources - it counts against the score, as S3 did in the worked example.
• No sources at all - an answer drawn from the model's training data comes with no context, so a groundedness check has nothing to measure it against.
• Refusals and filler - TruLens counts abstentions as grounded by default and drops trivial statements, and Google's API skips lines it judges to need no check. An answer that says little can score well, so pair groundedness with answer relevance.
• Input limits - Bedrock checks up to 100,000 characters of source, a 1,000-character query and a 5,000-character response, and Vertex AI accepts up to 200 facts of 10,000 characters each and answers of up to 4,096 tokens. The free character counter shows whether a long response fits before you send it to a check with a size limit.
• Chat and language scope - Amazon states that conversational QA and chatbot use cases are not supported by its check, and Microsoft's groundedness detection supports English content only.
• Claimed actions - an agent's report that it changed a record describes an event, and no passage retrieved before the change can document it.

Testing Claimed Actions That Have No Source Passage With Agent Assurance

Give the docs assistant one tool, a call that raises an API key's rate limit, and it can reply "Done, your key now allows 1,000 requests per minute." That sentence reports an event in an admin system, and no chunk retrieved before the call can show that the event took place. A false report of that kind is called an agent action hallucination, and only the state of the admin system after the call can expose it.

TestMu AI's Agent Assurance tests agents of that kind before release. It reads the agent's source code or spec to write scenarios, functional and adversarial ones by default and non-functional ones when you ask, then calls the real agent in staging or CI and decides each acceptance criterion from what the run produced:

• Tool calls - the calls the run made, matched to the tool list the agent itself declares, so a criterion such as "the limit tool is called and the delete-key tool is not" is decided by the calls themselves.
• Files and artifacts - each file changed under the paths your profile lists during the test, plus any artifact the agent saved.
• Records - a read-only lookup made with a tool you have approved, for example a query for the key's current limit. Aim test runs at staging, since the agent changes real data when it acts.
• Unable to Verify - when no evidence could settle a criterion, the report marks it Unable to Verify instead of Pass or Fail. Those criteria get a separate tally, and the pass rate is computed without them, so a gap in the evidence moves the result neither up nor down.

Because Agent Assurance runs before release, it does not filter answers in production the way Bedrock's threshold does. Still pre-alpha and publicly installable, it ships as Rook CLI, a command-line tool. Installing it from npm takes Node.js 20 or newer, and the second command below prints the installed version:

npm install -g @testmuai/rook
rook --version

Rook CLI can be installed on macOS and Linux, and 64-bit Windows through npm or WSL. For a Mac or Linux machine without Node.js, the Rook CLI installation guide gives a Homebrew command and a shell script instead. In CI, a finished run can exit 0 while scenarios fail, so gate the job on the JSON that rook run --json returns, its ok, halted and report.totals fields, as the docs page Run Agent Assurance in CI/CD describes.

How to Add Groundedness Checks to an Evaluation Pipeline

Start with the worked example's script: replace its sample claims with labels from answers in your own logs, and see how far the rules spread before you choose one. Then add the checks in this order:

• Write the rules down - the claim unit, sentence or single fact; whether a partly supported sentence fails; and whether a claim the context never mentions counts against the answer.
• Log the context each answer received - the exact chunks or tool responses that reached the prompt, because scoring against a fresh retrieval measures a different input.
• Label a calibration set by hand - real questions and answers labeled claim by claim; then run your checker on the same set and count where it disagrees with you.
• Store every score with its settings - the tool, the scale, the judge model, the library version and the count of unsupported claims, so a moved score points either to the answer or to the scorer.
• Test actions separately - for agents that change records, check the tool calls, files and records a run leaves behind. The Agent Assurance quickstart starts with a public sample, a local HTTP service with in-memory tickets that needs no model key.