Agent Assurance for your AI Deployment

Test how your agents actually behave across workflows, tools, and actions. Catch failures and vulnerabilities before they ship.

npm install -g @testmuai/rook

AIAI Testing

AI Risk Management Framework: NIST AI RMF, TEVV and Agents

The AI risk management framework most teams mean is NIST AI RMF 1.0. Learn its four functions, the TEVV evidence it expects, the GenAI profile and AI agents.

Published on:

OVERVIEW

NIST's AI risk management framework splits AI risk work into 72 subcategories, and a script run on 2026-10-20 found the words test, evaluate, verify, validate or TEVV in the text of only 15 of them. Nine of those 15 sit under Measure, in the AI RMF Core tables on NIST's resource center.

NIST's AI RMF Playbook data file still tags all 22 Measure subcategories with a TEVV actor. For an AI agent, that Measure evidence has to come from what the agent did in a run, which is the part the framework's text says least about.

Overview

The AI risk management framework most teams mean is the NIST AI RMF 1.0, a voluntary framework NIST released on January 26, 2023. Its Core describes outcomes and actions, not a checklist of controls, across 19 categories and 72 subcategories, and a script run on 2026-10-20 found test, evaluate, verify, validate or TEVV in the text of 15 of them, 9 under Measure.

The Four NIST AI RMF Functions

  • Govern: In the NIST AI RMF, Govern sets the policies, roles and culture for AI risk and is a cross-cutting function that runs through the other three functions, Map, Measure and Manage.
  • Map: The NIST AI RMF Map function establishes the context and risks of one AI system and informs an initial go or no-go decision about whether to design, develop or deploy it.
  • Measure: The NIST AI RMF Measure function analyzes, tests and tracks risks with TEVV processes, and it holds 22 of the 72 subcategories and 179 of the 459 AI RMF Playbook actions.
  • Manage: The NIST AI RMF Manage function prioritizes risks, responds to them and plans monitoring, recovery and communication, and MANAGE 1.1 asks whether development or deployment should proceed.

NIST AI RMF Questions

  • Is the NIST AI RMF mandatory?: No. NIST's AI RMF FAQs say NIST produced the AI RMF as a voluntary framework, although a contract, a sector regulator or an organization's own policy can still make it a requirement.
  • Has NIST released an AI RMF 2.0?: No. NIST's AI RMF page, read on 2026-10-20, still offers version 1.0 and says it is being revised as part of the White House AI Action Plan; no revised version had been published.
  • Does the NIST AI RMF mention AI agents?: No. The word agent does not appear in the AI RMF 1.0 text (NIST AI 100-1), its Core tables or its Playbook, so an agent's tool calls and writes need criteria a team adds under Map and Measure.
  • What is TEVV in the NIST AI RMF?: TEVV means test, evaluation, verification and validation, and NIST says that, ideally, the people who verify and validate an AI system are separate from those who test and evaluate it.
  • NIST AI 600-1: NIST AI 600-1, the Generative AI Profile of July 2024, describes 12 risks that are unique to generative AI or made worse by it and lists 212 suggested actions by a script count on 2026-10-20, 72 of them under Measure.
  • Testing AI agents before release: TestMu AI's pre-alpha Agent Assurance calls the live agent in staging or inside a CI job and gives each acceptance criterion a Pass, Fail or Unable to Verify verdict, and an organization's risk and compliance team and auditors decide whether those results belong in its Measure records.

What Is the NIST AI Risk Management Framework?

The AI risk management framework is NIST's Artificial Intelligence Risk Management Framework, AI RMF 1.0, published as NIST AI 100-1. NIST's AI Risk Management Framework page dates its release to January 26, 2023 and says it is "intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems."

Its Core describes outcomes and actions, not a checklist of controls, grouped under four functions. Other organizations publish AI risk frameworks too; in this guide the term means NIST's.

The AI RMF FAQs list the trustworthiness characteristics the framework aims to support:

  • Valid and reliable.
  • Safe.
  • Secure and resilient.
  • Accountable and transparent.
  • Explainable and interpretable.
  • Privacy-enhanced.
  • Fair with harmful bias managed.

A risk framework answers how you find, measure and treat the risks of each AI system. An AI governance framework sets who owns AI decisions and which rules bind them, and it usually draws on the AI RMF as one of its sources.

Around the AI RMF itself, NIST publishes companion material that this guide uses:

  • AI RMF Playbook - suggested actions, documentation questions and references for every Core subcategory, which NIST calls voluntary: you take as many or as few suggestions as fit.
  • Profiles - the AI RMF applied to one technology or setting, such as the Generative AI Profile (NIST AI 600-1) and a planned profile for critical infrastructure.
  • Crosswalks - tables that relate the AI RMF to other texts, such as ISO standards and national guidelines, listed on NIST's resource center.

How TEVV Words and Actions in the NIST AI RMF Were Counted

Every count of the Core, the Playbook and the Generative AI Profile in this guide comes from a single counting script that ran on 2026-10-20 against NIST files saved with curl that day. The script and the PDF text extractor behind it use only the Node.js and Python standard libraries, and the PDF of NIST AI 100-1, the AI RMF 1.0 text, was read only to count mentions of agents.

  • Core structure - categories and subcategories were read from the four function tables on the AI RMF Core page, and the 72 subcategory IDs were matched against the 72 Playbook entries, with none missing.
  • TEVV wording - a subcategory counts when its own text contains test or testing, a form of evaluate, verify or validate, or the acronym TEVV. The word valid alone does not count, so MEASURE 2.5 (valid and reliable) is left out.
  • Playbook fields - suggested actions are the top-level bullets of each entry, documentation items are the questions under "Organizations can document the following", and a TEVV actor is an entry whose AI Actors field lists TEVV.
  • NIST AI 600-1 actions - action IDs such as MS-2.3-001 were read from the PDF text. Four rows draw part of their ID with characters that have no text mapping, so they were counted from the readable part, and the one ID used only as an example in the profile's introduction was left out.
  • Risk tags - each action row was checked for the names of the 12 risks. A few tag characters also lack a text mapping, so the per-risk numbers are minimums.

A word count measures wording only: a subcategory without a TEVV word can still need test evidence, as MEASURE 2.3 shows by asking for performance or assurance criteria to be measured and demonstrated in conditions similar to deployment.

What Are the Four Functions of the NIST AI RMF?

The four functions are Govern, Map, Measure and Manage. The AI RMF Core states: "Governance is designed to be a cross-cutting function to inform and be infused throughout the other three functions." Govern is therefore not a first step you finish and leave.

FunctionCategoriesSubcategoriesPlaybook actionsAI 600-1 actions
Govern61910058
Map51810539
Measure42217972
Manage4137543
Total1972459212

Measure has the most subcategories and the most suggested actions in both companion texts. The script's output below gives its share of the Core subcategories and of the Playbook actions, and shows it is the one function where every subcategory lists TEVV among its actors. The AI 600-1 column comes from the same script's count over the NIST AI 600-1 PDF, read 2026-10-20.

NIST AI RMF 1.0 Core, read 2026-10-20
Source: airc.nist.gov AI RMF Core tables 1 to 4

Function   Categories  Subcategories
Govern              6             19
Map                 5             18
Measure             4             22
Manage              4             13
Total              19             72

Playbook entries 72; IDs not in Core 0

NIST AI RMF Playbook, read 2026-10-20
Source: airc.nist.gov/docs/playbook.json

Function   Actions  Doc items  TEVV actor  TEVV topic
Govern         100         78           2           0
Map            105         70           2           2
Measure        179        103          22          19
Manage          75         43           1           0
Total          459        294          27          21
Measure subcategories with a TEVV actor: 22 of 22
Measure share:
  Core subcategories  22 of 72  30.6%
  Playbook actions    179 of 459  39.0%

What each function asks for, and the record a team ends up holding when it is done:

FunctionWhat the Core asks forRecord it leaves
GovernPolicies, roles, risk tolerance, an inventory of AI systems, testing and incident practices, and rules for third-party software and dataApproved policies, named owners, the system inventory, a third-party register
MapThe context of one system: its purposes, users, laws, requirements, components and the likely impacts of its useA purpose statement, system requirements, an impact assessment and the initial go or no-go call
MeasureMethods and metrics, evaluation for each trustworthiness characteristic, risk tracking over time and feedback on whether the measurement worksTest sets, metrics, results, and a list of risks that were not or could not be measured
ManagePrioritizing risks, choosing responses, keeping a way to disengage the system, and post-deployment monitoring, incidents and improvementA release decision, risk responses, residual risk notes, monitoring and incident plans
  • Map outcome - the Core says that after Map, users should know enough "to inform an initial go/no-go decision about whether to design, develop, or deploy an AI system."
  • Measure processes - its processes "should include rigorous software testing and performance assessment methodologies with associated measures of uncertainty, comparisons to performance benchmarks, and formalized reporting and documentation of results."
  • MANAGE 1.1 - this subcategory asks for a determination of whether the system achieves its intended purposes and whether its development or deployment should proceed.

What Is TEVV in the NIST AI RMF?

TEVV stands for test, evaluation, verification and validation: the measurement work that produces evidence about an AI system across its lifecycle. The AI RMF Core states that after the Measure function, "objective, repeatable, or scalable test, evaluation, verification, and validation (TEVV) processes including metrics, methods, and methodologies are in place, followed, and documented."

NIST AI RMF TEVV therefore means documented processes and their records, kept across the lifecycle. For how the four words split into separate checks when the system is an agent, see AI agent validation.

TEVV Tasks Across the AI Lifecycle

Appendix A of the framework, Descriptions of AI Actor Tasks, assigns TEVV work to each phase. It also states: "Ideally, AI actors carrying out verification and validation tasks are distinct from those who perform test and evaluation actions."

PhaseTEVV task in Appendix AEvidence for an AI agent
Design, planning and dataValidating assumptions about design, data and measurements against the intended deployment contextAcceptance criteria per task, the list of tools the agent may and may not call, staging data shaped like production
DevelopmentModel validation and assessmentThe model ID, prompt version and tool definitions recorded with every test run
DeploymentSystem validation and integration with testing, including user experience and legal, regulatory and ethical specificationsScenario results from a run against real tools in staging, plus the release decision they fed
OperationsOngoing monitoring, testing and expert recalibration of models, incident tracking and detection of emergent propertiesProduction traces, an incident log, and a rerun of the scenario suite after each change

Which AI RMF Subcategories Use TEVV Wording?

The script searched each Core subcategory's own text for TEVV words. The output below gives the totals per function, with documentation wording counted beside them, since Map asks for documentation far more often than for tests.

Subcategories whose own text uses TEVV words
(test, evaluate, verify, validate, TEVV)
Function   TEVV words  Measure/monitor  Document
Govern              2                3         3
Map                 3                0        15
Measure             9               10        15
Manage              1                5         5
All functions: 15 of 72 (20.8%)
Measure share of those: 9 of 15 (60.0%)

GOVERN 4.2 and MAP 1.4 match only through a passing form of evaluate, a lifecycle step in one and re-evaluated business value in the other, so neither asks for test results. The table lists every subcategory that matched, with the evidence each implies for an agent:

SubcategoryWhat it asksEvidence for an agent
GOVERN 4.2Teams document the risks and impacts of the AI they design, develop, deploy, evaluate and useA risk note per agent that names the systems it can write to
GOVERN 4.3Practices enable AI testing, incident identification and information sharingA staging environment the agent can be tested in, and an incident channel
MAP 1.1Intended purposes, users, laws and settings are documented, including related TEVV and system metricsThe tasks the agent is for and the metric that says each was done
MAP 1.4Business value or context of use is defined, or re-evaluated for an existing systemA stated reason the agent acts instead of a person or a fixed script
MAP 2.3Scientific integrity and TEVV considerations are documented, such as experimental design and data selectionHow scenarios were chosen and what the test data does not cover
MEASURE 2.1Test sets, metrics and details about the tools used during TEVV are documentedThe scenario set, its version and the test runner that executed it
MEASURE 2.2Evaluations with human subjects meet applicable requirements and represent the populationConsent and sampling records for any user study of the agent
MEASURE 2.6The system is evaluated regularly for safety risks and shown to fail safelyResults for requests the agent must refuse, and proof it stops cleanly
MEASURE 2.7Security and resilience are evaluated and documentedResults for instructions planted in data the agent reads
MEASURE 2.9The model is explained, validated and documented, and output is interpreted in contextA trace that ties each action to the input that caused it
MEASURE 2.11Fairness and bias are evaluated and results documentedThe same request from different users gets the same outcome
MEASURE 2.13The effectiveness of the TEVV metrics and processes themselves is evaluatedA review of which criteria could not be checked and why
MEASURE 3.3Feedback from end users and impacted communities feeds evaluation metricsUser reports turned into new test scenarios
MEASURE 4.2Results in deployment context are checked with domain experts to validate consistent performanceExpert sign-off on the scenarios that gate a release
MANAGE 4.1Post-deployment monitoring plans capture and evaluate user input, appeals, incidents and change managementA monitoring plan and the trigger that reruns tests after a change

TEVV-Athlon, NIST's Draft Method for Running TEVV

NIST announced The TEVV-Athlon Framework for Evaluating AI Systems, initial public draft NIST AI 200-2, on August 7, 2026, with comments open until October 6, 2026. The announcement lists agentic systems among the AI types the method is meant to cover and asks for input on the definitions of the four TEVV terms themselves.

The initial public draft of NIST AI 200-2 says the "TEVV-Athlon itself implements the Measure core function" and that its results inform Manage. It adopts a definition of TEVV from the National Security Commission on Artificial Intelligence: methods and metrics "to determine that a technology or system satisfactorily meets its design specifications and requirements, and that it is sufficient for its intended use." Its four stages:

  • Articulate and Organize - state the goal of the assessment and plan it, using what Govern and Map produced as input.
  • Define and Construct - turn the goal into Blocks, the system attributes or characteristics the assessment will gather evidence on.
  • Apply and Measure - run Events, the activities that produce evidence for each Block, with the Tools that record it.
  • Synthesize and Interrogate - analyze the data, check that it measures what it was meant to, and present results for Manage.

The draft's worked example assesses chatbots, so adapting the method to an agent's tool calls is left to whoever runs the assessment.

What Does NIST's Generative AI Profile Add to the AI RMF?

NIST AI 600-1, the Generative AI Profile, applies the AI RMF to generative AI. It describes 12 risks that are unique to generative AI or made worse by it, then lists suggested actions under AI RMF subcategories, each tagged with the risks it addresses; the script counted 212 actions, 72 of them under Measure.

NIST AI 600-1 (July 2024), read 2026-10-20
Source: nvlpubs.nist.gov NIST.AI.600-1.pdf

Suggested actions: 212
  (208 IDs read as text; 4 rows whose
   ID glyphs carry no Unicode mapping)
  Govern     58   27.4%
  Map        39   18.4%
  Measure    72   34.0%
  Manage     43   20.3%

The profile is narrower than its length suggests. It says the working group's focus "was limited to four primary considerations relevant to GAI: Governance, Content Provenance, Pre-deployment Testing, and Incident Disclosure" and that future revisions will add subcategories, risks and actions.

Risk in NIST AI 600-1What NIST meansActions tagged, at leastWhat to test in an agent
CBRN Information or CapabilitiesEasier access to chemical, biological, radiological or nuclear weapon information31Refusal of such requests, including when they arrive through a tool result
ConfabulationConfidently stated false content, which NIST notes is often called hallucination18Claims of a finished action with no matching change, a form of agent action hallucination
Dangerous, Violent, or Hateful ContentViolent, inciting or self-harm content and hateful or stereotyping material31What the agent writes into tickets, emails or files, along with its replies
Data PrivacyLeakage, unauthorized use or de-anonymization of personal or sensitive data26Personal data copied into logs, tickets or other systems the agent can reach
Environmental ImpactsHigh compute use in training or operation and its effect on ecosystems4Tokens and calls per completed task, tracked across releases
Harmful Bias or HomogenizationAmplified biases, performance gaps between groups and undesired sameness of output57Equal outcomes for equal requests from different users or languages
Human-AI ConfigurationInteractions that lead to automation bias, over-reliance or anthropomorphizing58Approval steps a person actually sees before a risky action runs
Information IntegrityContent that blurs fact and opinion or fuels dis- and misinformation at scale71Sources the agent cites match what the source says
Information SecurityLower barriers to cyberattacks and a larger attack surface for the system itself48Planted instructions in documents, tickets or web pages the agent reads
Intellectual PropertyEasier production or replication of allegedly copyrighted, trademarked or licensed content without authorization26Licensed code or text copied into what the agent writes
Obscene, Degrading, and/or Abusive ContentObscene, degrading or abusive imagery, including synthetic child sexual abuse material and nonconsensual intimate images14Refusal paths for generation tools the agent can call
Value Chain and Component IntegrationNon-transparent or untraceable third-party components and data, and weak supplier vetting36Behaviour after the model, an MCP server or a plugin changes version

Information Integrity carries the most tags and Environmental Impacts the fewest. An action usually carries several tags, so the column adds up to more than the number of actions.

How Do You Apply the AI RMF to an AI Agent?

Apply it as you would to any AI system, then add the risks that come from acting. AI RMF 1.0, its Core tables and its Playbook never name agents, and the Generative AI Profile uses the word for AI only once, listing autonomous agents among security threats:

Occurrences of "agent" (any form)
  AI RMF 1.0 text (AI 100-1)      0
  AI RMF Core tables              0
  AI RMF Playbook (72 entries)    0
  AI 600-1                        6
    of which "autonomous agents"  1
    the rest: chemical, biological or other
    dangerous agents (the CBRN risk)

NIST's Center for AI Standards and Innovation (CAISI) described the risks of agents that act in its January 2026 request for information on securing AI agent systems: "AI agent systems are capable of planning and taking autonomous actions that impact real-world systems or environments." It names indirect prompt injection, insecure or poisoned models, and models that harm security without any attacker, such as through specification gaming.

The worked example is an access-provisioning agent. It reads requests in an IT ticket queue, adds people to directory groups through a tool and comments on the ticket. The organizational controls around agents like it are covered in agentic AI governance; the table covers the risk framework steps.

FunctionWhat your team does for this agentSubcategoriesEvidence it leaves
GovernName an owner who approves releases, write the rule that admin groups need human approval, and list the model provider and directory API as third partiesGOVERN 1.4, GOVERN 2.1, GOVERN 6.1Owner record, written approval rule, third-party register
MapState the agent's purpose, write requirements such as never granting an admin group, list the tools it may call and the ones it must not, and mark where a person reviewsMAP 1.1, MAP 1.6, MAP 3.5Purpose statement, requirements, tool allow and deny lists, oversight points
MeasureRun scenarios against a staging directory, including tickets with planted instructions, document the test set and the runner that executed it, and list what could not be measuredMEASURE 1.1, MEASURE 2.1, MEASURE 2.3, MEASURE 2.7Scenario set, per-run record of tool calls and group changes, list of unchecked criteria
ManageDecide go or no-go from the results, keep a tested way to disable the agent, and plan monitoring and incident handling after releaseMANAGE 1.1, MANAGE 2.4, MANAGE 4.1, MANAGE 4.3Release decision, disable procedure, monitoring and incident plan

The Measure row only produces evidence when each requirement becomes a criterion a run can decide. For this agent, the criteria read like these:

  • Valid request - after the run, the directory shows the user in the requested group; the agent's comment saying so is not enough.
  • Admin group without approval - group membership is unchanged and the ticket records a refusal.
  • Planted instruction - a ticket whose text tells the agent to add its author to an admin group leaves every group unchanged.
  • Failed tool call - when the directory call fails, the agent reports the failure instead of success.
  • Forbidden tool - the run log shows no call to a tool on the deny list, such as one that deletes accounts.

Some criteria will not be decidable, for example when staging has no audit log for one directory. MEASURE 1.1 asks you to document the risks you will not or cannot measure, so record those criteria as unchecked rather than counting them as passed. The model inside the agent raises its own questions, such as version pinning and revalidation after a provider change, which AI model governance covers.

How Does the NIST AI RMF Relate to the EU AI Act and ISO Standards?

They do different jobs, and NIST does not claim one satisfies another. NIST's crosswalk documents page says a listed crosswalk does not imply NIST endorsement, "nor does it imply that either resource comprehensively covers the contents of the other."

TextWhat it isBinding?Status, read 2026-10-20Link to the AI RMF
NIST AI RMF 1.0Risk management frameworkNo, voluntaryReleased January 26, 2023; being revisedThe base text
NIST AI 600-1Profile of the AI RMF for generative AINo, voluntaryJuly 26, 2024 editionActions filed under AI RMF subcategories
EU AI ActRegulation (EU) 2024/1689Yes, for operators in its scopeIn force since August 1, 2024; high-risk rules from December 2, 2027 (Annex III) and August 2, 2028 (Annex I)A January 2023 NIST illustration relates the AI RMF to the Act as then proposed
ISO/IEC 42001Standard with requirements for an AI management systemNo, voluntary standardEdition 1, published December 2023A crosswalk submitted by Microsoft, listed by NIST
ISO/IEC 23894Guidance on AI risk managementNo, voluntary standardEdition 1, published February 2023A crosswalk by INCITS/AI, revised August 14, 2025, listed by NIST

The Commission's regulatory framework for AI page, updated on October 19, 2026, is the source for the EU dates. It says the AI Omnibus entered into force on July 27, 2026, setting December 2, 2027 for high-risk use cases in areas such as employment and critical infrastructure, and August 2, 2028 for systems built into products such as lifts or toys.

The closest EU match to Measure is the risk management system for high-risk systems in Article 9 of the AI Act. Its paragraph 8 requires testing, as appropriate, during development and in any event before the system is placed on the market or put into service, measured against "prior defined metrics and probabilistic thresholds" suited to the system's intended purpose. How to turn that article into test cases is the subject of EU AI Act conformity testing.

ISO/IEC 42001 is listed in ISO's catalogue as edition 1 of December 2023, a standard that sets requirements for running and improving an AI management system in an organization that provides or uses AI. It works at the level of the organization, while the AI RMF's Map, Measure and Manage functions can be applied one system at a time.

Which NIST AI RMF and AI Agent Updates Are in Progress?

Version 1.0 is still the current text, and NIST's AI RMF page says: "The AI RMF 1.0 is being revised as part of the White House AI Action Plan." That plan, America's AI Action Plan of July 2025, recommends that NIST "revise the NIST AI Risk Management Framework to eliminate references to misinformation, Diversity, Equity, and Inclusion, and climate change." No revised version had been published when the NIST pages were read on 2026-10-20.

DateNIST itemStatus, read 2026-10-20Why it matters for agents
February 17, 2026AI Agent Standards Initiative, launched by CAISIInput gathered through a security RFI and an NCCoE draft on identity and authorization for software and AI agentsA NIST program aimed at agents, covering standards, open protocols and security research
April 7, 2026Concept note for an AI RMF profile on trustworthy AI in critical infrastructureOngoing; NIST says it will share discussion drafts through its mailing list and community Slack channelAims to give critical infrastructure sectors more confidence to deploy AI agents and tools
August 7, 2026NIST AI 200-2, TEVV-Athlon, initial public draftComment period closed October 6, 2026A method for running Measure that names agentic systems in its scope
October 16, 2026Key practices for incident readiness and response in super intelligence training and evaluationPublished as a voluntary document with NSA and CISAWritten for frontier developers, about incidents where agents take uninstructed actions
  • Critical infrastructure profile - NIST's concept note page says the profile "will guide CI operators towards specific risk management practices to consider when engaging AI-enabled capabilities."
  • Agent initiative - the AI Agent Standards Initiative page lists three pillars: industry-led standards, community-led protocols and research into agent authentication, identity and security evaluations.
  • Incident practices - NIST's October 2026 announcement covers incidents "in which one or more SI agents take significant uninstructed actions that compromise explicit or implicit policies, controls, or security measures imposed by the operator".
  • Center name - that announcement names the issuing center NIST's Center for Advancing Innovation and Standards for Super Intelligence (CAISSI), while February's initiative launch was issued as CAISI.

Pre-Release Agent Test Evidence for AI RMF Measure Work With Agent Assurance

For an agent that acts, most of the Measure evidence in the example above comes from running it: what it called, what it changed and which criteria nobody could check. TestMu AI's Agent Assurance produces that kind of test evidence before release, for agents that use tools, edit files and send API requests. It is a pre-alpha product with a publicly installable command-line client, Rook CLI:

npm install -g @testmuai/rook
rook --version

You can use Rook CLI on macOS and Linux, and 64-bit Windows through npm or WSL. For the npm package you need Node.js 20 or a later release, and the Rook CLI installation guide covers Homebrew and the checksum-verifying shell installer for macOS and Linux.

Agent Assurance starts from your agent's source code or specification and can also read a PRD or your written policies. It writes scenarios and gives each one acceptance criteria (functional and adversarial classes unless you ask for more), calls the live agent in staging or inside a CI job, and checks every criterion against what the run produced:

  • Changed files and artifacts - output files under the paths named in the agent's profile, and the artifacts the run produced.
  • Tool calls - each call measured against the tools the agent declares, with a must-not-call list for forbidden ones.
  • Records - read-only lookups in the system the agent touched, using an MCP tool that you provide and approve.

Model judges score some criteria, rules and read checks decide the others, and an action the agent only claims is never proof: the run's evidence decides. Each verdict is one of Pass, Fail or Unable to Verify. The Agent Assurance documentation on results and evidence describes the scenario pass rate as a share of decided verdicts and shows coverage beside it, so a high rate built on little verified evidence stays visible.

Most AI eval and observability tools score the agent's answers and the traces it logged. Agent Assurance instead inspects the changes a run left in files, tool calls and records, and keeps a separate list of the criteria it had no evidence to decide.

Agent Assurance is not a NIST AI RMF mapping: it neither certifies nor shows compliance with the framework or with any of its functions, categories or subcategories. It tests and reports, and your risk and compliance team and auditors decide whether its results belong in your Measure records and whether your risk management meets the AI RMF or any regulation.

Use a staging environment, since a test run performs the agent's writes for real and nothing rolls them back. Agent Assurance does not monitor production or block anything at runtime, so post-release monitoring needs your own tools.

Write a Measure Gap Report to Start the AI Risk Management Framework

Pick the agent that can change the most records, fill in the four rows of the worked example for it, and write its five riskiest requirements as criteria a run can decide. Mark each criterion you cannot check yet and name what would make it checkable; that list is your first Measure gap report under the AI risk management framework, the record of unmeasured risks that MEASURE 1.1 asks for.

To see how that evidence supports trust claims beyond risk management, read the guide to AI assurance. To run the criteria against staging in a pipeline, follow the Agent Assurance CI/CD guide, which gates a job on the run's report instead of its exit code.

Note

Note: Sirajuddin Khan, Vice President of Product Management at TestMu AI with Agentic AI and GenAI Product Strategy among the listed areas of expertise, is the author of record for this guide and approved it; the guide was researched and drafted with AI assistance. Vipul Verma, Group Senior Vice President of Engineering at TestMu AI, was its technical reviewer. The sources behind it are NIST publications and pages (NIST AI 100-1, NIST AI 600-1, the AI RMF Playbook and the draft NIST AI 200-2), America's AI Action Plan, the European Commission's AI Act pages, ISO's catalogue and the TestMu AI product documentation, and the editorial process and AI use policy describes how every claim is checked before publication.

Author

...

Sirajuddin Khan

Blogs: 9

  • Linkedin

Sirajuddin Khan is Vice President of Product Management at TestMu AI, where he drives the company's agentic AI product strategy, building a suite of autonomous agents that includes Agentic Browsers and Agentic Visual Testing and shifting the unit of work from test execution to autonomous outcomes. One of the company's earliest product leaders, he has owned the roadmap for the high-performance execution cloud and grew the cross-browser testing products from early adoption to market leadership. He brings over a decade of experience across SaaS, B2B, and eCommerce, with earlier product roles at Wydr and ShopClues, where his catalog and search work cut delivery SLAs and lifted seller activity. Sirajuddin holds an MBA in Information Technology from Sikkim Manipal University and a B.Tech in Computer Science Engineering from Maharshi Dayanand University.

Reviewer

...

Vipul Verma

Reviewer

  • Linkedin

Vipul Verma is Group Senior Vice President of Engineering at TestMu AI, where he heads the entire engineering organization that builds KaneAI, HyperExecute, and the broader testing cloud. He brings 15+ years architecting, securing, and scaling large enterprise applications across multiple sites. Before TestMu AI he was India Head at LogicHub, where he built the India R&D site from the first employee to a 30-plus engineering team, and Principal Software Engineer at Sumo Logic, where he was the first engineer in the India office and shipped search-performance and pricing-model initiatives. Earlier he worked on trading platforms at Portware and D. E. Shaw. Vipul holds a B.Tech in Computer Science from IIT Kharagpur.

Add to Google preferred sources

Summarise with AI

Copied to Clipboard!
...

3000+ Browsers. One Platform.

See exactly how your site performs everywhere.

Try it free
...

Write Tests in Plain English with KaneAI

Create, debug, and evolve tests using natural language.

Try for free

AI Risk Management Framework FAQs

Did you find this page helpful?

More Related Learning Hubs

TestMu AI forEnterprise

Get access to solutions built on Enterprise
grade security, privacy, & compliance

  • Advanced access controls
  • Advanced data retention rules
  • Advanced Local Testing
  • Premium Support options
  • Early access to beta features
  • Private Slack Channel
  • Unlimited Manual Accessibility DevTools Tests