AI Agent Validation: How to Validate Agents Before Release

AI Agent Validation: Definition and What Counts as Evidence

AI agent validation is the confirmation, through objective evidence, that an AI agent meets the requirements of one specific intended use: a named job, for named users, in a named environment. It produces a release decision and a record of the evidence behind it, and it is repeated whenever the agent or its job changes.

The definition follows quality-management wording that predates AI agents. The NIST AI Risk Management Framework 1.0 (January 2023) states that validation is the "confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled", and gives ISO 9000:2015 as its source.

Each part of that sentence (ISO 9000:2015, as quoted by NIST) sets a task for the team releasing an agent:

• A specific intended use - validation is always of an agent for a job. The same agent can be valid for changing delivery addresses and not valid for issuing refunds.
• Requirements - the job has to be written down as acceptance criteria that a run can meet or miss.
• Objective evidence - the result has to rest on something a second person could inspect and reach the same conclusion from.

For an agent that acts, objective evidence is anything observed from outside the agent's own account:

• The state of the system of record after the run - the order, account, file or message the job was meant to change, read directly.
• Tool calls captured by the runtime - which tools were called, with which arguments, and what each returned, logged by the platform the agent runs on.
• Files and artifacts - what was written to disk or produced as output, with its location and type.
• Measured values - latency, token usage and cost taken from usage logs.
• Human review against a written rubric - a reviewer's recorded judgment on cases a rule cannot decide.

The agent's final message, a transcript of its reasoning and a judge model's score of either one are useful signals, and all three are the agent's own account or a reading of it. On their own they do not validate an action.

AI Agent Validation Compared With Evaluation, Verification and Testing

The four words answer four different questions. Validation is the last of them and the one that carries a signature.

Testing is running the agent on cases and comparing what happened with what should have happened. The guide to AI agent testing covers how to write scenarios and score an agent's responses, and validation uses those runs as raw material.

Evaluation turns many runs into numbers: a completion rate, a tool-use score, a judge model's rating. AI agent evaluation explains the scoring methods, and a high score still says nothing about whether the test set resembles the job.

Verification checks against something already written down. The NIST CSRC glossary entry for verification gives the ISO 9000:2015 wording: "Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled." The two definitions differ in a few words, "specified requirements" for verification and "a specific intended use or application" for validation, and AI agent verification covers checking an agent's identity, its authority and a single claimed action.

Validation uses the results of the other three. An agent can pass its tests, score well and meet every written requirement, and still be unfit for the job when the requirements missed something the job needs.

This guide is about agents that act: they call tools, write files and change records. If the agent you are releasing holds conversations instead, TestMu AI's Agent Testing covers chat, voice, phone (inbound and outbound), video and image agents.

Why Can an AI Agent That Passes Synthetic Tests Fail on Real Work?

Synthetic cases are written by people who know what the agent was built to handle, so they share its assumptions. Real work arrives with missing fields, contradictory instructions, stale records and tools that time out.

The JAMIA Open study in the introduction measured that difference on one system. Its abstract calls the result a "reality gap" with performance drops of 11 to 32 percentage points between the synthetic and the real-world figures. Its conclusion is that "synthetic validation alone provides false confidence", and it sets a requirement: "Rigorous real-world ground truth evaluation with expert annotation is essential before clinical deployment."

NIST makes the same observation about generative AI in general. Its Generative AI Profile (NIST AI 600-1), published in July 2024, warns that the testing done before deployment today may not match the context a system is deployed into, and it lists that under the limits of current pre-deployment test approaches.

A synthetic test suite makes an agent look better than it is in these ways:

• Clean inputs - every request is complete, in one language, and about one thing.
• Tools that always succeed - a mocked tool returns the expected response, so the agent's handling of errors, timeouts and partial writes is never exercised.
• No history - each case starts from an empty state, while a real account carries earlier orders, open disputes and half-finished changes.
• A check that reads the reply - the case passes when the final message says the task is done.

Validation adds a realistic environment (real requests, real tool behavior and realistic account state, in staging) and evidence taken from outside the agent, so a case passes on what the records show.

What Should You Write Down Before You Validate an AI Agent?

Write the validation plan before the first run, because every item in it is harder to state honestly once you have seen results. A plan for one agent fits on a page:

• Intended-use statement - two or three sentences that name the tasks, the users, the systems the agent may change and the tasks it must hand to a person. Example: "Change delivery addresses and cancel subscriptions for signed-in customers. Refunds go to a person."
• Risk tier - what the worst plausible mistake costs, and whether it can be undone. An agent that changes an address is in a lower tier than one that moves money.
• Acceptance criteria with thresholds - each criterion measurable, each with a number, each tagged blocking or advisory.
• Environment and data - where the runs happen, which tools are live, where the test accounts come from, and how many cases are drawn from real requests.
• Named approver - one person, by name, who owns the risk of the job and did not build the agent.

The plan should also state what the validation will not cover. NIST's AI RMF 1.0 asks for this in its MEASURE 2.5 outcome: "The AI system to be deployed is demonstrated to be valid and reliable. Limitations of the generalizability beyond the conditions under which the technology was developed are documented."

For an agent, those limits are the edges of the intended use: languages not tested, customer types not sampled, tools not connected in staging. Anything outside them is not validated, whatever the pass rate inside them.

How to Validate an AI Agent in Seven Steps

The steps run in this order because each one fixes something the next depends on: the job before the criteria, the criteria before the cases, the cases before the runs, and the evidence before the decision.

Three Sources of Evidence for One Agent Action: the Report, the Trace and the System of Record

Step 5 needs the most care, because three things can each look like proof that an action happened. Take one action, a customer's request to change the delivery address on an open order, and compare what each source can show.

Suppose the agent replies that the address is updated, and the trace shows one call to the address-update tool that returned success. The order record, read a minute later, still carries the old address, because the order had already been released to the warehouse and the update applied to future orders.

Two of the three sources say the task succeeded, and the parcel still goes to the wrong house. The guide to agent action hallucination covers the ways an agent comes to report work it did not complete.

GitHub measured how often one computer-use agent judged its own runs correctly. In Validating agentic behavior when "correct" isn't deterministic (GitHub Blog, 6 May 2026), the agent's self-assessment, tested in GitHub's own test suite, reached 82.2% accuracy and 60.0% recall at telling successful executions from failed ones. The authors report that the agent "frequently misreported failures as successes" and conclude that "agents cannot yet reliably grade their own homework in non-deterministic environments."

The script below was run on 8 October 2026 (Node.js v25.5.0) to show how the choice of evidence source changes a release decision. Its input is sample data: an imaginary order-support agent, an imaginary validation run against four acceptance criteria, and an imaginary earlier sign-off.

The three blocking criteria are each counted twice, once as the agent's final messages reported them and once as a check of the records found them. The fourth, cost, is measured once from usage logs.

node release-check.mjs

SAMPLE DATA: release check for an imaginary agent

INTENDED USE
Change delivery addresses and cancel subscriptions
for signed-in customers. Refunds go to a person.

ACCEPTANCE CRITERIA (set before testing)
A1 End state correct (blocking), need >= 95%
   agent reports     60/60 = 100.0%   MET
   records checked   56/60 = 93.3%    NOT MET
A2 No forbidden action (blocking), need 0
   agent reports     0 in 40 runs     MET
   records checked   0 in 40 runs     MET
A3 Hands off when required (blocking), need >= 90%
   agent reports     20/20 = 100.0%   MET
   records checked   18/20 = 90.0%    MET
A4 Cost per finished task (advisory), need <= $0.08
   usage logs        $0.05            MET

CHANGED SINCE THE SIGN-OFF OF 2026-09-12
model        yes   rerun the full suite
prompt       no
tools        yes   rerun cases that call the tool
knowledge    no
permissions  yes   rerun forbidden-action cases

DECISION
On the agent reports alone:  RELEASE
On the records checked:      HOLD (A1 not met)
Sign-off: withheld

None of these figures describes a real agent, and the thresholds and the blocking and advisory tags are examples for illustration, so set your own from the task and its risk. The output shows how a validation record behaves:

• The decision flips on where the evidence comes from. On the agent's reports alone, every blocking criterion is met and the decision is RELEASE. On the records, criterion A1 is 56 of 60 runs (93.3%) against a threshold of 95%, and the decision is HOLD.
• In the sample, four runs in sixty were reported as done and were not. The agent's reports show 60 of 60.
• 93.3% is close to 95% and it is still NOT MET. A1 is a blocking criterion and its threshold was set before testing, so nobody gets to argue it down after the run.
• Criterion A3 is met on both sources although they differ: 20 of 20 by the reports, 18 of 20 by the records. A gap between the two sources is worth reading even when the threshold holds.
• The sign-off of 2026-09-12 does not cover this release candidate. The model, one tool and the permissions changed after it, and the sample plan attaches a rerun scope to each kind of change.

Criterion A2 shows a different limit. Zero forbidden actions in 40 adversarial runs meets the rule as written, and 40 runs cannot show that a forbidden action will never happen.

How Many Runs an AI Agent Needs Before a Pass Counts

An AI agent's behavior varies from run to run, so one green run is an anecdote. The same case, with the same input and the same configuration, can end in the correct state on Monday and the wrong one on Tuesday.

State a validation result so that the variation shows:

• State every result as a rate with its run count - "56 of 60 runs ended in the correct state" can be checked and compared. "The agent passes" cannot.
• Decide the number of runs in the plan - the riskier the criterion, the more runs it needs, and that number is set before the first run for the same reason the threshold is.
• Treat zero failures in a small sample as evidence, not proof - a rare failure needs many runs before it shows up once, so a zero-tolerance criterion calls for the largest run count in the plan and a second control, such as a permission the agent does not hold.

The arithmetic is in two other guides: pass@k vs pass^k explains the difference between an agent that succeeds at least once in k attempts and one that succeeds every time, and testing non-deterministic AI outputs covers how many runs a given level of confidence takes.

How to Validate a Multi-Agent System: Each Agent, Each Handoff and the Whole System

When several agents share a job, validate at three levels, and expect different failures at each:

• Each agent - give every agent its own intended-use statement and criteria, and validate it alone with the other agents replaced by fixed inputs. Observe its actions in the systems it writes to.
• Each handoff - check what one agent passes to the next: whether the message is complete, whether the receiver acts on it or re-derives it, and what happens when the sender reports a success that did not occur.
• The whole system - run end-to-end cases and check the final state of every system of record. Observe duplicated actions, loops between agents, and tasks that every agent assumed another one had finished.

The system level cannot be skipped by adding up the first level. A July 2026 survey, Beyond Component Testing: Validating Agentic AI Systems (arXiv 2607.29405, version 1 of 31 July 2026), screened 7,197 retrieved records down to 257 included papers and assigned multi-agent validation as the primary dimension of 50 of them. The common conclusion of that work, in the survey's words, is that "collective failures and interaction-level hazards cannot be inferred from single-agent scores alone."

The survey is a preprint and its counts may change in later versions. For designing the cases at each level, see the guide to multi-agent testing.

Revalidating an AI Agent: Changes That Void a Sign-Off

A sign-off describes one configuration on one date. The arXiv survey cited above names the failure that follows from forgetting this, evidence decay: "a system once judged acceptable no longer merits the same claim after its operating context changes."

The same survey says deployments need explicit conditions for "what kinds of change invalidate prior claims" and "what revalidation scope follows", and it calls turning monitoring signals into revalidation triggers an unsolved problem. Until that problem is solved, write your own triggers into the validation record. The table is a starting set.

The sample output earlier in this guide shows three of these changes: the model, one tool and the permissions changed, and its sample plan gave each a scope. The table is the fuller version. How to build and run the rerun itself is the subject of agent regression testing.

Between sign-offs, production signals tell you when a trigger has fired without anyone announcing it, and the guide to AI agent monitoring lists those signals.

Acceptance Criteria You Can Grade Against Evidence With Agent Assurance

The hard part of every step above is checking the action as well as the answer. An agent that acts can return a correct-looking reply while the record behind it was never written, and a validation that reads replies has validated the agent's account of its work.

Suppose the job in your intended-use statement includes cancelling a subscription on request. In a staging run the agent tells the customer the subscription is cancelled, while the billing record still lists it as active with a renewal date.

TestMu AI's Agent Assurance is built for agents that act, and it runs your agent against staging before it ships. It derives test scenarios from the agent's code, or from its PRD or spec, and you supply how to invoke the agent. Each scenario carries acceptance criteria, and each criterion is graded on its own against evidence:

• A criterion about a tool - graded from the tool calls your profile hands back after the run, compared with the tools the agent declares. A forbidden action in your plan becomes a call that must be absent.
• A criterion about a file - graded from what changed on disk inside the output paths you declare in the profile.
• A criterion about a record - graded with a read-only query through a tool you provide and approve. In the example, the query returns the subscription's status from the billing record.
• Three verdicts - every criterion gets Pass, Fail or Unable to Verify, and Unable to Verify is not a failure. The pass rate is the share of passed scenarios among those with a decided verdict, so an unverified result neither raises nor lowers it. Read it beside verification coverage, the share of criteria that could be verified.

Rerun against staging whenever a change could alter the agent's behavior. The report gives the pass-rate change between your last two runs and which scenarios newly fail or were fixed.

You can also run it as a release gate in CI on a reviewed branch. There, gate on the verdicts in rook report --json, not on the exit code.

What a run can observe is set by your profile and by the access you grant. If the profile hands back the agent's answer and nothing else, most criteria come back Unable to Verify.

Most eval and observability tools score what your agent said and recorded. Agent Assurance checks what the run changed, and reports what it could not verify.

A claimed action never counts as proof in Agent Assurance. Because its runs happen before release, it is not a guardrail or a runtime monitor.

Agent Assurance is pre-alpha and publicly installable, and it runs from the terminal as Rook CLI. Install it with npm:

npm install -g @testmuai/rook

Rook CLI supports macOS and Linux, and 64-bit Windows through npm or WSL; npm is the install route that needs Node.js 22 or newer. If you work in Claude Code, one command adds the rook skill:

npx @testmuai/rook-skill@latest install --agent claude-code

The docs page on Agent Assurance test scenarios shows how acceptance criteria are written, and a second guide explains how to run Agent Assurance in CI/CD. Runs perform real writes, so aim each one at a staging environment.

AI Agent Validation Checklist for Release Sign-Off

Before the approver signs, every question below should have a yes. A no is either a reason to hold the release or an exception written into the record.

• Is the intended use written down, including the actions the agent must never take and the tasks it hands to a person?
• Were the acceptance criteria and thresholds agreed and dated before the first run?
• Is each criterion tagged blocking or advisory?
• Did the runs use the release configuration: the same model version, prompt, tools, knowledge sources and permissions?
• Did the runs happen in staging against the tools the agent will use after release?
• Does the case set include cases drawn from real requests?
• Was each case run more than once, and is every result stated as a rate with its run count?
• Was every action criterion checked in the system of record, and is the evidence source written beside each result?
• Does every blocking criterion meet its threshold on that evidence?
• Has each failed run been reviewed for what it would cost in production and given a decision?
• Is the approver named, and independent of the team that built the agent?
• Does the record list the changes that void the sign-off and the rerun scope for each?

The agent's own statement that it is ready is not on the list. In Agentic Uncertainty Reveals Agentic Overconfidence (arXiv 2602.06948, a preprint of February 2026), researchers asked coding agents to estimate their own chance of success on 100 SWE-Bench Pro tasks across three models, and found that "some agents that succeed only 22% of the time predict 77% success." That figure measures a prediction of success on coding tasks, which is a different thing from a completion report, and it points the same way as GitHub's figures.

The validation record is the document the approver signs. Keep it to the parts an outsider would need to repeat the decision:

• Scope - the intended-use statement and the limits of what was validated.
• Configuration - model version, prompt version, tool versions, knowledge sources and permissions.
• Environment - where the runs happened and which tools were live.
• Runs - the number of cases, the number of runs per case, and where the cases came from.
• Results - each criterion with its threshold, its rate and run count, and the source of the evidence.
• Exceptions - each failure, its materiality and the decision taken on it.
• Decision - release or hold, the approver's name and the date.
• Expiry - the changes that void the sign-off and the rerun scope each one triggers.

Start with one agent and one job. Write its intended-use statement and three or four acceptance criteria, name the approver, and run the first validation in staging with the results counted from the records.