AI Agent Assurance for Your Agent Deployments
Agent Assurance Tests How Your Agent Actually Behaves across Workflows, Tools, and Actions to Catch Failures and Vulnerabilities.
npm install -g @testmuai/rook@latest
RAG Evaluation Grades the Answer - Test the Chain of Custody Instead
RAG evaluation that only scores answers misses leaks, stale data and bad retrieval. Use the 5-link RAG Chain of Custody to test RAG before customers do.
Published on:
A lot of the RAG evaluation out there asks just one question: was the answer good? You run a question set, you get an LLM to score faithfulness and relevance, and you're good to go when the numbers look okay.
That will catch some of the hallucinations. It won't catch the kind of failure that ends up in the incident reviews: the assistant that quoted a policy that had in fact been retired the previous quarter, the assistant that pulled a document the user wasn't actually allowed to see, or the assistant that cited a page which in reality didn't say what the answer claimed.
Our view at TestMu AI is actually pretty straightforward. A RAG answer is a claim with a chain of evidence behind it. You go through the chain link by link and keep the evidence at each link.
The same split between the answer and the evidence behind it runs through the Microsoft AI Transparency Initiative, where the tool call is the record.
The scale of this is urgent. In Menlo Ventures' 2024 enterprise survey, RAG was at 51% adoption in enterprise AI deployments (up from 31% the year prior).
Retrieval is now the default way that enterprises ground their models in their own data. It's not something that merits a score.
TL;DR
RAG evaluation should be testing the chain of custody behind the answer rather than just the answer itself. The RAG Chain of Custody tests five links in sequence (Corpus, Access, Retrieval, Attribution, Answer), uses deterministic assertions for the first four and then saves the model-based judgment for the final one, with the judge being a different entity from the generator.
What Are the Key RAG Evaluation Metrics?
- RAG triad: The triad is the core set of RAG evaluation metrics: faithfulness, answer relevance and context relevance, each graded against whatever context was retrieved. Model-graded: yes. The triad cannot tell whether that context was current or permitted.
- Ragas: Ragas is an open-source library that computes the triad scores with an LLM grader; the Ragas metrics reference lists each score, and the RAG evaluation tools roundup compares it with other scorers.
- Corpus freshness: The indexed version of every source-of-truth document must match the live version, and every retired document must return zero hits for any query. Deterministic: yes.
- Abstention: A golden set needs unanswerable questions, and the system must abstain or escalate on them instead of inventing an answer. Deterministic: yes, scored against a labeled set like the rest of your RAG testing.
- Claim-to-citation matching: Every citation must point to a retrieved chunk, and the numbers, dates and named entities in the answer must appear in that chunk. Deterministic: mostly, with a model used only for paraphrase entailment.
- Independent judge: Model-based grading belongs only on the Answer link, using a judge from a different model family than the generator, calibrated on a human-labeled anchor set. Any LLM evaluation or AI evals program that grades with a model needs the same split.
How Do You Test a RAG System for Data Leakage?
- Persona access tests: Build personas from real identity groups such as contractor or finance lead, ask each the same questions, and block the release if a restricted document appears in retrieved results, citations or answer text.
- Audit evidence: Store the question, persona, retrieved IDs and versions, citations, answer and verdicts in systems you control, so every run doubles as AI verifiability evidence for an AI audit.
- Pre-release AI assurance: Production observability scores traffic after release, while pre-release AI assurance records what was tested before users see it. TestMu AI's Agent Testing runs persona-driven conversations against chat and voice RAG assistants.
Why Isn't the RAG Triad Adequate for RAG Evaluation?
The triad will assess the response against whatever context the system has retrieved. It has absolutely no way of knowing whether that context was the right one, the current one, or the appropriate one.
The standard metrics are from the RAGAS paper (Es et al., EACL 2024 demos): faithfulness, answer relevance and context relevance. The authors explicitly made them reference-free so that the teams could assess without having to go to the trouble of human-labeled ground truth, and the prompts were scored by an LLM (gpt-3.5-turbo-16k).
That was the right call to make to get going. It has two consequences that very few teams ever go back and look at.
Firstly, reference-free means you can still get a very high faithfulness score by answering the wrong document. If the retrieval has pulled the wrong document (e.g., last year's refund policy) and the model simply regurgitates that accurately, you can still end up with a very high faithfulness score, and the customer is still completely in the dark.
Secondly, model-graded metrics are susceptible to model bias. Panickssery, Bowman and Feng (NeurIPS 2024) found that GPT-4 could distinguish its own outputs from those of two other LLMs and from human outputs 73.5% of the time, and there was a linear correlation between self-recognition and self-preference. If your generator and your grader are from the same model family, your RAG evaluation metrics are likely to be in part measuring familiarity.
Retrieval is not a safety net at all. Google Research in May 2025 found that Gemma's wrong answer rate went from 10.2% with no context to as high as 66.1% with insufficient context. RAG actually reduced the model's tendency to just say 'I don't know'.
The underlying sufficient context paper (ICLR 2025) found that 55.4% of the instances in one standard benchmark were actually missing sufficient contextual information, and frontier models were still getting the right answer 35-62% of the time when the context was insufficient.
Answer-only scoring makes no difference between a lucky answer and a grounded one.
And even production-grade retrieval fails. The Stanford RegLab and HAI study (which was subsequently published in the Journal of Empirical Legal Studies) found that the top AI legal research tools from LexisNexis and Thomson Reuters each hallucinated between 17% and 33% of the time (so less than a general-purpose chatbot, but by no means zero). Retrieval does help to reduce the error rate, though it doesn't prove anything on its own.

What Is the RAG Chain of Custody?
The RAG Chain of Custody is a RAG evaluation framework that treats every answer as the end of a five-link chain (Corpus, Access, Retrieval, Attribution, Answer), tests the links in that order, and keeps the evidence for each one.

The rule that makes it work is that a link is only as good as the link before it. A perfect answer score on top of an untested Access link is not a pass. It's a risk that hasn't been tested and has been awarded a good grade.
| Link | What it checks | How it's graded | Example assertion | Evidence kept |
|---|---|---|---|---|
| 1. Corpus | Is the right content indexed, current, and is retired content gone? | Deterministic | Indexed version of each source-of-truth doc matches the live version; retired docs return zero hits | Doc IDs, version hashes, index timestamp |
| 2. Access | Does retrieval respect the asker's permissions? | Deterministic | Persona "contractor" never retrieves or cites anything in the restricted set | Persona, retrieved IDs, allow/deny result |
| 3. Retrieval | Does the right passage come back, and does "no answer" come back when there is none? | Deterministic against a labeled set | Expected passage in top k; unanswerable questions trigger abstention | Ranked IDs, scores, abstain flag |
| 4. Attribution | Does every claim trace to a cited passage that says it? | Mostly deterministic | Every citation points to a retrieved chunk; numbers and dates in the answer string-match the cited text | Claim-to-passage map |
| 5. Answer | Is it correct, complete and appropriately cautious? | Independent judge + human anchor set | Judge from a different model provider agrees with the human-labeled set above your agreed rate | Judge model, verdict, rationale |
Four of the five links don't require a model to grade them. That's the point.
Any assertion that can be made deterministically should be so. A deterministic check will always produce the same result and can be carried out by someone who doesn't have any faith in your model.
How Do You Test Each Individual Link in a RAG Pipeline?
Test each link with its own assertion, starting upstream so that if there's a failure you can attribute it to the stage which caused it rather than just have a general dip in the quality of your answers.
Link 1: Corpus, the Link Nobody Owns
Corpus failures are the dullest of the bunch and probably the easiest to spot. You've got content that's been updated in the source system but not re-indexed, or content that's been removed from the wiki but still exists in the vector store. Write your freshness tests in the same way as you would write your contract tests.
For every document that is the source of truth, you assert that the indexed version is identical to the live version. For every document that's been retired, you assert that it can't be retrieved by any query. They're flash tests that fail in a very obvious manner.
Link 2: Access, Where Deployer-Defined Authority Becomes Testable
One of the AI trust principles we have at TestMu AI is deployer-defined authority (it's the company that decides what the model is allowed to see and do, not the model itself). In the RAG context, that principle is either going to be respected or not at retrieval time.
The OWASP Top 10 for LLM applications 2025 has Vector and Embedding weaknesses as LLM08. The first two risks on that list are unauthorized access and data leakage, and cross-context information leakage between users or queries.
Vendors treat that as a moving part. Glean's release notes from 3rd November 2026, for instance, included a check that a subscriber can see a Slack channel prior to an agent sending a message into it. The permission logic is constantly changing on your side and on theirs.
So test it as a moving part. Create personas from your actual identity groups (intern, contractor, regional manager, finance lead). Ask the questions of each of those personas.
Be absolutely clear that no restricted documents will ever appear in the retrieved set (neither in the citations nor in the answer text). That's the one instance where the pass threshold is zero failures.

Link 3: Retrieval, Including the Right to Say "I Don't Know"
Classic retrieval tests (is the expected passage in the top k?) are all fine and dandy, and there are loads of guides on that. What's generally missing is the negative case. With the sufficient context in the findings above, every golden set will require some unanswerable questions (questions the corpus simply can't answer), and we need to be very clear that the system either abandons or escalates rather than just inventing an answer.
Link 4: Attribution, Where AI Verifiability Gets Concrete
A citation is a commitment that the source says precisely what the answer says. Check the mechanical bits deterministically: the citation is present, it refers to a chunk that was actually retrieved, and all the numbers, dates, and named entities in the answer are present in that chunk. You can only use a model for paraphrase entailment, and you must use one from a different provider than your generator.
Link 5: Answer, Judged by Someone Else
This is exactly where an LLM as a judge comes in, and only in this scenario. Two rules: the judge must be from a different model family from the generator (i.e., model diversity, no model judging itself) and the judge must be compared against a human-labeled anchor set, which you re-score every time the judge model changes.
RAG Evaluation Approaches Compared
Answer-only scoring and the RAG triad are useful development tools, neither of which in any way will give you the evidence that you'd need for a release gate or an audit.
| Approach | Catches | Misses | Who grades | When it runs | Evidence for audit |
|---|---|---|---|---|---|
| Answer-only spot checks | Obvious wrong answers | Lucky answers, leaks, stale content | Humans, ad hoc | Before launch | Little |
| RAG triad metrics (RAGAS-style) | Ungrounded answers, irrelevant context | Wrong-but-faithful answers, permission leaks | An LLM, often the generator's family | Development, CI | Scores without provenance |
| Production observability and evals | Drift and failures on real traffic | Problems before release | Platform judges on live traces | After release | Traces, owned by the platform |
| RAG Chain of Custody | All five links, with failures located to a stage | Novel attacks outside the test set | Deterministic checks + independent judge | Every change that can move the chain, at the release gate | Per-link records kept in your systems |
Observability and pre-release testing are absolutely complementary rather than competitive. Security and observability companies are both building up their production evaluation portfolios at a very rapid rate.
CrowdStrike announced on 3rd November 2026 that it was acquiring Fiddler for agent observability (that's the post-release stuff). Chain of Custody is the pre-release stuff; it's the stuff you can prove.
How Do You Run Your RAG Evaluation as Part of Your Release Gate?
Run the chain on every change that can move it, block the release on any Access failure, and have the per-link evidence stored where your auditors (not your vendor) have control of it.
A practical checklist for QA leaders:
- Golden set with three question types: answerable, unanswerable and forbidden for this persona.
- Personas mapped to real identity groups rather than to invented roles.
- Triggers defined: re-index, chunking or embedding change, prompt change, model version change, permission or connector change. Each one re-runs the chain.
- Thresholds set from your own baseline for Corpus, Retrieval, Attribution and Answer. Access remains at zero tolerance.
- Judge independence recorded: generator provider, judge provider and the anchor set version all logged for each run.
- Evidence stored in your control plane: question, persona, retrieved IDs and versions, citations, answer, verdicts. Human-readable, tamper-evident and held to your audit policy.
- A named owner for each link. The corpus is generally owned by the content or platform teams. Access to IAM, Retrieval and Attribution is via the AI team (that's the gate to QA).
Does This Apply if We'd Bought Our RAG Rather Than Built It?
Yes, you can't inspect the vendor's ranker, but you can test every link through the interface that your users actually use. In either case, the onus for the answer is still yours.
In Moffatt v. Air Canada (Feb 2024), the BC Civil Resolution Tribunal found Air Canada was responsible for all of the information on its website and dismissed the airline's argument that the chatbot in any way took responsibility for the statements it made. The award was trivial at $812.02. The principle is not.
If you built the assistant or you licensed it, you're responsible.
For bought systems, you run the chain black box (i.e. personas through the real UI or API and assertions on what comes back). Keep the evidence yourself rather than just taking it from the vendor dashboard. That's our independent audit principle in action: an audit shouldn't be dependent on the system under audit.
For in-house systems, the edge case is organizational. The team that built the pipeline shouldn't be the ones that are the ones who grade it for exactly the same reason that developers won't approve their own PRs. You need to separate out the people who build the pipeline from the people who grade it.
Where Does Agent Assurance Fit Into RAG Evaluation?
RAG is increasingly going to be one step in a larger agent workflow, so Chain of Custody becomes one element of pre-release Agent Assurance rather than a standalone report.
At TestMu AI, we think of software interaction in three broad categories and RAG features in two of those. A human asking a chat or voice assistant is the Human<->Agent scenario. Our Agent Testing runs all those persona-driven conversations and checks what the agent actually said and cited.
When an agent calls an agent (i.e., a retrieval agent) with no human in the loop, that's the Agent<->Agent scenario, which we call Agent Assurance, and that's delivered by Rook. SDETs can wire in chain checks into their CI via the Kane CLI and run the full persona x question matrix in parallel at the release gate on HyperExecute.
We hold ourselves to the same rules: our verification is entirely independent of the model under test, is based on deterministic assertions and leaves a trail of evidence for the customer to keep.

What Will Have Changed by the End of 2027?
Our prediction: by the end of 2027, enterprise reviews of RAG assistants will be asking for per-persona access evidence as well as quality scores.
Three trends in that direction. OWASP already has access leakage from vector stores on their list of the top LLM risks.
Gartner's survey (published 16th November 2026) of 298 infrastructure and operations leaders found that 95% had at least one agent-related incident over the past 12 months and 70% felt their agents were operating within safe boundaries.
That's the Gartner survey of AI agents in general (not RAG specifically), and the gap between what they believe and what the evidence shows is exactly the same gap that RAG teams are confronting.
In Europe, the Commission's draft fundamental rights impact assessment template notes that deployers in scope will be required to comply from 2nd December 2027. The assessments will be based on evidence, not scores.
Conclusion: RAG Evaluation Is a Custody Problem
NIST AI 600-1 defines confabulation as confidently stated but false content. In the RAG world, confidence is cheap.
The citation looks legitimate, the tone is very calm, the score is green. The only real answer is proof of provenance (where it came from), proof of who was allowed to see it, and proof of who actually checked it.
So change the question that your RAG evaluation is asking. Instead of "was the answer good?" it should be "can we show the chain behind it?" Test the corpus, the access, the retrieval, and the attribution before you even test the answer itself with a judge who didn't write it.
Keep the evidence. That's what takes a RAG demo and turns it into something an enterprise can actually trust.
Shipping agents that other agents use? Rook by TestMu AI does Agent Assurance: agents testing agents before release, with no human in the loop.
npm install -g @testmuai/rook@latest command, or for a full walkthrough of the product.Author
Mudit Singh is Co-Founder and Head of Growth at TestMu AI, and a member of the founding team that has grown the platform to 2.8M+ users across 132 countries. He drives growth and go-to-market across the full product portfolio, including KaneAI, Kane CLI, the Real Device Cloud, SmartUI, accessibility testing, and HyperExecute. Over more than eight years on the founding team he has helped build and bring these testing products to market from scratch, scaling the company from cross-browser testing into a full-stack agentic AI quality engineering cloud backed by Sequoia, Qualcomm Ventures, and Premji Invest. He brings over 15 years of experience building and marketing software products and holds a B.Tech in Computer Science from Jaypee Institute of Information Technology.
Reviewer
Mayank Bhola is Co-Founder and Head of Products at TestMu AI, where he leads the entire product portfolio across KaneAI, Kane CLI, HyperExecute, SmartUI, the Real Device Cloud, Accessibility, and other software testing product lines. As an early Lead Architect he designed and built the company's flagship Tunnel technology from scratch, created the React-based automation platform, and architected the data-intensive pipelines and FAAS services that scale it. He brings more than 10 years of experience in software development and product engineering, with earlier roles as Head of Technology at Juggernaut Books and Senior Software Engineer at PressPlay TV and Zomato. Mayank holds a B.Tech in Computer Engineering from JIIT Noida.
FAQs
Did you find this page helpful?
More Related Blogs
TestMu AI forEnterprise
Get access to solutions built on Enterprise
grade security, privacy, & compliance
- Advanced access controls
- Advanced data retention rules
- Advanced Local Testing
- Premium Support options
- Early access to beta features
- Private Slack Channel
- Unlimited Manual Accessibility DevTools Tests

















