What Is AI Verifiability?

What Does AI Verifiability Mean?

AI verifiability is the share of an AI system's behavior that someone other than the system can confirm from evidence. For an AI agent it is assessed one requirement at a time: a record from outside the agent's reply shows the requirement was met, shows it was missed, or does not exist.

The definition rests on verification in its standards sense. Among the definitions that NIST's glossary lists for verification, the one taken from ISO 9000:2015 describes it as confirming, through objective evidence, that specified requirements were fulfilled.

Researchers, standards bodies and vendors use verifiability in AI for different properties:

• Governance principle - the AI Ethics Lab at Rutgers University describes verifiability as the principle that an AI system's functionality, decision-making processes and outputs can be "independently validated and confirmed", backed by documentation, repeatability and independent audits. In Toward Trustworthy AI Development (2020), Miles Brundage and colleagues argued that AI developers "will need to make verifiable claims to which they can be held accountable", and analyzed ten mechanisms for supporting such claims across institutions, software and hardware.
• Cryptographic provenance - the W3C Verifiable Credentials Data Model v2.0, a W3C Recommendation since May 15, 2025, describes a verifiable credential as "a specific way to express a set of claims made by an issuer, such as a driver's license or an education certificate". A signature over those claims shows who made them and that nobody changed them; whether an agent did its job is a separate question.

The rest of this guide uses the operational sense, AI agent verifiability, which a team can measure on its own agent. Checking whether one agent is who it claims to be, is allowed to act and did what it reported is AI agent verification; verifiability is how much of that checking the agent makes possible.

Why Are Verifiable Tasks Easier for AI to Automate?

Andrej Karpathy tied verification to automation in his essay Verifiability, dated November 17, 2025. Writing about what he calls the new programming paradigm, he paired two lines: "Software 1.0 easily automates what you can specify." and "Software 2.0 easily automates what you can verify." In his account a task is verifiable when an AI can practice it, and "Tasks that are verifiable progress rapidly".

His conditions for the environment in which an AI practices, and what each becomes when the task is testing an agent that acts:

The third column is this guide's mapping; the essay does not discuss agents, permissions or releases. Applying the idea to deployment is TestMu AI's own argument: grant an agent write access only where a test run can confirm, before release, how the agent uses it. Where no run can confirm that yet, keep a person approving each such action, or hold the permission back.

In Software 2.0 Means Verifiable AI, written for O'Reilly Radar on December 9, 2025, Mike Loukides warned that verification "might prove to be more difficult than training LLMs". For a team shipping agents, that means budgeting for the checks as well as the agent; why checking now lags behind generating is covered in the guide to the verification agent.

How Is AI Verifiability Different From Explainability, Observability and Auditability?

For an AI system that acts, each property asks a different question and relies on different evidence:

A model can explain an action it never took; verifiability looks at the action.

• Explainability - NIST's AI Risk Management Framework defines explainability as "a representation of the mechanisms underlying AI systems' operation". An agent's reasoning trace is often read as that kind of explanation; how agents come to report work they did not do is covered in agent action hallucination.
• Observability - AI observability records traces, logs and metrics to show what happened and where it failed, mostly in production and also in CI experiments. A recorded tool call is evidence, so better traces raise verifiability, and you can measure the gain before release.
• Auditability - a reviewer rebuilds decisions and controls later from retained records, and what an agent's audit trail should hold is covered in agentic AI governance. An AI audit can only conclude on the evidence it is handed, so the verifiability built in before release sets how much behavioral evidence an audit can draw on.

How Do You Measure AI Verifiability?

The AI verifiability metric in this guide, verification coverage, is counted per acceptance criterion. A scenario pass rate divides the scenarios that passed by those that reached a Pass or Fail verdict, so it cannot show how many acceptance criteria any evidence settled.

AI assurance checks a system against its written criteria and reports what the evidence shows; verifiability sets how much evidence there is to report.

8 decided scenarios: 8 passed, 0 failed
20 acceptance criteria: 8 verified, 12 unable to verify

Scenario pass rate: 100%
Criterion verification coverage: 40%

OWASP red-team coverage (scenario inventory; unmapped
  controls remain unverified)
risk | generated | executed | reached | verified | Pass |
  Fail | Unable to Verify
redteam | 4 | 0 | 0 | 0 | 0 | 0 | 4
owasp:asi01 | 1 | 0 | 0 | 0 | 0 | 0 | 1
owasp:llm01 | 1 | 0 | 0 | 0 | 0 | 0 | 1
owasp:llm02 | 2 | 0 | 0 | 0 | 0 | 0 | 2
owasp:llm07 | 1 | 0 | 0 | 0 | 0 | 0 | 1
...
Known coverage gaps (unverified):
F-001 · cross-session-memory-poisoning: no scenario
  generated; applicability remains unverified
F-003 · retrieval-boundary: no scenario generated;
  applicability remains unverified

What Makes an AI Agent More Verifiable?

An agent's verifiability depends on what its runs leave behind, so it can be designed in. The Agent Assurance overview also describes the low end: "A profile whose hook returns only an answer string leaves most criteria Unable to Verify".

For a CRM agent like the one above, each action is as verifiable as the record it leaves outside its reply:

Rules 1 and 3 to 6 below each match a fix in the docs' Fix Verification Gaps table, on the results page cited above; rule 2 relies on the calls that rule 1 records:

• Return each tool call with its arguments and result - have the hook that runs the agent report the calls it made, read from a log the agent cannot edit. The Rook CLI docs page on profiles and hooks lists a calls field for observed tool calls, and the results page warns: "Hooks must return actual observations. Invented usage or calls turn missing evidence into misleading scores."
• Declare allowed and forbidden tool calls - write them into each scenario's expectations, including the calls the agent must not make, so an unexpected call becomes a detectable failure as long as a hook reports the calls.
• Give the checker read-only access to what the agent changes - confirm the billing contact by reading account 4471 back, never by repeating the write. Where a check would itself write, the docs' fix is to "Provide a read-only observation instead."
• Collect evidence that arrives late - an email receipt or a trace can land after the agent replies. For delayed traces or files, the docs' fix is a collect hook, after which the same run continues with the --run option.
• Write each criterion as a check a record can settle - "the account's billing contact equals the address in the request" can be verified, while "updates customer records correctly" cannot. When a criterion expects a field the evidence lacks, the docs say to align the two without silently weakening the business requirement; deciding what counts as the right answer at all is the test oracle problem.
• Isolate fixtures and keep staging resettable - give each scenario its own accounts and invoices so evidence can be attributed to it, run one scenario at a time when scenarios share state, and restore staging between runs.

Rules 1 and 2 also need work in the agent harness, the code around the model that runs its tools and keeps its state: it has to log every call to a place the agent cannot edit, and hold the list of permitted tools. Rules 3 to 6 change only the test setup and the criteria.

How Do You Close Verification Gaps Before Release?

Treat the first coverage number as a baseline: each criterion it leaves unverified comes with a reason you can act on. Repeat this loop until the number stops rising:

• Run one suite against staging - use synthetic accounts and invoices, isolated as in design rule 6.
• List every unverified criterion with its reason - group them by cause, such as missing tool calls, late evidence or a check that would itself write.
• Close each gap with the matching design rule - change what the agent records or what the checker can read, and leave the requirement as written.
• Rerun the same scenarios - keep the scenario definitions and profile revision fixed so coverage is compared like for like, and run unchanged cases more than once before treating a change in results as a regression, since the agent may choose a different route through the same task.

Then decide what blocks a release. The Agent Assurance CI/CD guide publishes recipes for Argo CD, GitHub Actions and Jenkins that use a strict gate: missing evidence, Unable to Verify, incomplete work or a compromised result blocks the job, and blocking never relabels Unable to Verify as Fail. The general CI recipe in the public Rook CLI skill for coding agents is more permissive: it reports Unable to Verify and unrunnable gaps without failing on them alone.

If the strict gate blocks every release while coverage is still low, gate on Fail first, measure coverage for a few weeks, and only then add a coverage threshold the data supports. The guide also says to enforce any criterion-level evidence or coverage policy you need from the saved verdicts, because scenario totals alone do not prove that every criterion was observable.

Coverage tells you how much of a release decision rests on evidence. Whether the agent suits the job it was built for is what AI agent validation decides.

Testing How Much of an Agent's Work You Can Verify With Agent Assurance

Built by TestMu AI, Agent Assurance tests agents that act in other systems, like the CRM agent above, before release. It runs from your terminal as Rook CLI, the rook command, and works from the agent's code or, when there is no code to read, from its spec. From either, it generates functional and adversarial scenarios and reaches the real agent through whatever your profile names, whether an HTTP endpoint, a command or an MCP tool.

• Replies - what the agent said is graded on quotes taken from its actual output, and some criteria are scored by a model judge over the exchange.
• Actions - what the agent did is graded on evidence returned by the profile's hooks, for example the tool calls it made, a state change or a callback it triggered, and on read-only lookups through a tool you supply and approve, such as reading account 4471 back from the CRM.
• Unsupported claims - an action the agent reports with nothing behind it stays Unable to Verify, and the evidence a run can see is limited by the access and the profile you provide.
• Verdict counts - criteria are settled as Pass or Fail, or left Unable to Verify, and that third verdict keeps a tally of its own that is never merged into the other two. Only scenarios that reached a Pass or Fail enter the pass rate.
• Run records - every run is saved as plain files holding the scenarios exactly as that run used them, the pinned agent version and profile revision, and a verdict with quoted evidence for each criterion. Its summary is computed from those saved verdicts without a model call.

Most eval and observability tools grade an agent's replies and the traces its runs leave behind. Agent Assurance grades what a run changed, from the evidence your hooks report and the read-only checks you approve.

To get verification coverage, count the criterion verdicts in the verdict.yaml file that each scenario keeps in the run's folder, under .testmuai/rook in your project, or filter each result's acceptance criteria by Pass, Fail or Unable to Verify in the local or hosted UI. The totals in rook report --json count scenarios, so on their own they only show how many planned scenarios reached a decided verdict; criterion counts come from the verdict files.

Point Rook CLI at staging with synthetic records: the agent's writes are real, and Rook CLI cannot roll them back. The testing happens before release, and nothing in it watches production traffic or stops an agent while it runs.

Agent Assurance is generally available and self-serve; its free trial lasts 14 days and comes with 4,000 credits. To run, Rook CLI checks your TestMu AI account for an Agent Assurance plan or a trial in progress, and without one it refuses to call a model or spend credits; what each plan includes is on the Agent Assurance plans page.

Install Rook CLI with npm, and add the skill if you work in Claude Code:

npm install -g @testmuai/rook@latest
npx @testmuai/rook-skill@latest install \
  --agent claude-code

Supported systems are macOS and Linux, and 64-bit Windows through npm or WSL. Installed with npm, Rook CLI needs Node.js 20 or newer and the Claude Code skill needs Node.js 22 or newer; Homebrew and the shell installer, for macOS and Linux, need no Node. For the commands behind each route, see the Rook CLI installation guide.

TestMu AI was named a Leader in the 2026 Gartner® Magic Quadrant™ for Agentic Software Quality Assurance Platforms.

Start Measuring AI Verifiability With One Agent and One Suite

Pick one agent that writes to a system you can restore, such as the CRM agent above, and write five to ten of its acceptance criteria as checks against records. With Rook CLI, the criteria sit in the generated scenario files, which you can review and edit before a run. Run those scenarios against staging, divide the criteria settled with a Pass or Fail by all criteria, and report that number next to the pass rate.

To practice on a known target first, the Agent Assurance quickstart walks through the public triage-service sample from connecting the agent to reading its report. Discovery, generation and judging in that walkthrough spend credits.

Gartner, Magic Quadrant for Agentic Software Quality Assurance Platforms, By Ross Power, Sushant Singhal, C.A. Swan, 12 October 2026.

Gartner and Magic Quadrant are trademarks of Gartner, Inc. and/or its affiliates.

Gartner does not endorse any company, vendor, product or service depicted in its publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner publications consist of the opinions of Gartner's business and technology insights organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this publication, including any warranties of merchantability or fitness for a particular purpose.