Agent Assurance for your AI Deployment
Test how your agents actually behave across workflows, tools, and actions. Catch failures and vulnerabilities before they ship.
npm install -g @testmuai/rook
- TestMu AI (Formerly LambdaTest)
- /
- Learning Hub
- /
- AI Risk Management Framework: NIST AI RMF, TEVV and Agents
AI Risk Management Framework: NIST AI RMF, TEVV and Agents
The AI risk management framework most teams mean is NIST AI RMF 1.0. Learn its four functions, the TEVV evidence it expects, the GenAI profile and AI agents.
Published on:
OVERVIEW
NIST's AI risk management framework splits AI risk work into 72 subcategories, and a script run on 2026-10-20 found the words test, evaluate, verify, validate or TEVV in the text of only 15 of them. Nine of those 15 sit under Measure, in the AI RMF Core tables on NIST's resource center.
NIST's AI RMF Playbook data file still tags all 22 Measure subcategories with a TEVV actor. For an AI agent, that Measure evidence has to come from what the agent did in a run, which is the part the framework's text says least about.
Overview
The AI risk management framework most teams mean is the NIST AI RMF 1.0, a voluntary framework NIST released on January 26, 2023. Its Core describes outcomes and actions, not a checklist of controls, across 19 categories and 72 subcategories, and a script run on 2026-10-20 found test, evaluate, verify, validate or TEVV in the text of 15 of them, 9 under Measure.
The Four NIST AI RMF Functions
- Govern: In the NIST AI RMF, Govern sets the policies, roles and culture for AI risk and is a cross-cutting function that runs through the other three functions, Map, Measure and Manage.
- Map: The NIST AI RMF Map function establishes the context and risks of one AI system and informs an initial go or no-go decision about whether to design, develop or deploy it.
- Measure: The NIST AI RMF Measure function analyzes, tests and tracks risks with TEVV processes, and it holds 22 of the 72 subcategories and 179 of the 459 AI RMF Playbook actions.
- Manage: The NIST AI RMF Manage function prioritizes risks, responds to them and plans monitoring, recovery and communication, and MANAGE 1.1 asks whether development or deployment should proceed.
NIST AI RMF Questions
- Is the NIST AI RMF mandatory?: No. NIST's AI RMF FAQs say NIST produced the AI RMF as a voluntary framework, although a contract, a sector regulator or an organization's own policy can still make it a requirement.
- Has NIST released an AI RMF 2.0?: No. NIST's AI RMF page, read on 2026-10-20, still offers version 1.0 and says it is being revised as part of the White House AI Action Plan; no revised version had been published.
- Does the NIST AI RMF mention AI agents?: No. The word agent does not appear in the AI RMF 1.0 text (NIST AI 100-1), its Core tables or its Playbook, so an agent's tool calls and writes need criteria a team adds under Map and Measure.
- What is TEVV in the NIST AI RMF?: TEVV means test, evaluation, verification and validation, and NIST says that, ideally, the people who verify and validate an AI system are separate from those who test and evaluate it.
- NIST AI 600-1: NIST AI 600-1, the Generative AI Profile of July 2024, describes 12 risks that are unique to generative AI or made worse by it and lists 212 suggested actions by a script count on 2026-10-20, 72 of them under Measure.
- Testing AI agents before release: TestMu AI's pre-alpha Agent Assurance calls the live agent in staging or inside a CI job and gives each acceptance criterion a Pass, Fail or Unable to Verify verdict, and an organization's risk and compliance team and auditors decide whether those results belong in its Measure records.
What Is the NIST AI Risk Management Framework?
The AI risk management framework is NIST's Artificial Intelligence Risk Management Framework, AI RMF 1.0, published as NIST AI 100-1. NIST's AI Risk Management Framework page dates its release to January 26, 2023 and says it is "intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems."
Its Core describes outcomes and actions, not a checklist of controls, grouped under four functions. Other organizations publish AI risk frameworks too; in this guide the term means NIST's.
The AI RMF FAQs list the trustworthiness characteristics the framework aims to support:
- Valid and reliable.
- Safe.
- Secure and resilient.
- Accountable and transparent.
- Explainable and interpretable.
- Privacy-enhanced.
- Fair with harmful bias managed.
A risk framework answers how you find, measure and treat the risks of each AI system. An AI governance framework sets who owns AI decisions and which rules bind them, and it usually draws on the AI RMF as one of its sources.
Around the AI RMF itself, NIST publishes companion material that this guide uses:
- AI RMF Playbook - suggested actions, documentation questions and references for every Core subcategory, which NIST calls voluntary: you take as many or as few suggestions as fit.
- Profiles - the AI RMF applied to one technology or setting, such as the Generative AI Profile (NIST AI 600-1) and a planned profile for critical infrastructure.
- Crosswalks - tables that relate the AI RMF to other texts, such as ISO standards and national guidelines, listed on NIST's resource center.
How TEVV Words and Actions in the NIST AI RMF Were Counted
Every count of the Core, the Playbook and the Generative AI Profile in this guide comes from a single counting script that ran on 2026-10-20 against NIST files saved with curl that day. The script and the PDF text extractor behind it use only the Node.js and Python standard libraries, and the PDF of NIST AI 100-1, the AI RMF 1.0 text, was read only to count mentions of agents.
- Core structure - categories and subcategories were read from the four function tables on the AI RMF Core page, and the 72 subcategory IDs were matched against the 72 Playbook entries, with none missing.
- TEVV wording - a subcategory counts when its own text contains test or testing, a form of evaluate, verify or validate, or the acronym TEVV. The word valid alone does not count, so MEASURE 2.5 (valid and reliable) is left out.
- Playbook fields - suggested actions are the top-level bullets of each entry, documentation items are the questions under "Organizations can document the following", and a TEVV actor is an entry whose AI Actors field lists TEVV.
- NIST AI 600-1 actions - action IDs such as MS-2.3-001 were read from the PDF text. Four rows draw part of their ID with characters that have no text mapping, so they were counted from the readable part, and the one ID used only as an example in the profile's introduction was left out.
- Risk tags - each action row was checked for the names of the 12 risks. A few tag characters also lack a text mapping, so the per-risk numbers are minimums.
A word count measures wording only: a subcategory without a TEVV word can still need test evidence, as MEASURE 2.3 shows by asking for performance or assurance criteria to be measured and demonstrated in conditions similar to deployment.
What Are the Four Functions of the NIST AI RMF?
The four functions are Govern, Map, Measure and Manage. The AI RMF Core states: "Governance is designed to be a cross-cutting function to inform and be infused throughout the other three functions." Govern is therefore not a first step you finish and leave.
| Function | Categories | Subcategories | Playbook actions | AI 600-1 actions |
|---|---|---|---|---|
| Govern | 6 | 19 | 100 | 58 |
| Map | 5 | 18 | 105 | 39 |
| Measure | 4 | 22 | 179 | 72 |
| Manage | 4 | 13 | 75 | 43 |
| Total | 19 | 72 | 459 | 212 |
Measure has the most subcategories and the most suggested actions in both companion texts. The script's output below gives its share of the Core subcategories and of the Playbook actions, and shows it is the one function where every subcategory lists TEVV among its actors. The AI 600-1 column comes from the same script's count over the NIST AI 600-1 PDF, read 2026-10-20.
NIST AI RMF 1.0 Core, read 2026-10-20
Source: airc.nist.gov AI RMF Core tables 1 to 4
Function Categories Subcategories
Govern 6 19
Map 5 18
Measure 4 22
Manage 4 13
Total 19 72
Playbook entries 72; IDs not in Core 0
NIST AI RMF Playbook, read 2026-10-20
Source: airc.nist.gov/docs/playbook.json
Function Actions Doc items TEVV actor TEVV topic
Govern 100 78 2 0
Map 105 70 2 2
Measure 179 103 22 19
Manage 75 43 1 0
Total 459 294 27 21
Measure subcategories with a TEVV actor: 22 of 22
Measure share:
Core subcategories 22 of 72 30.6%
Playbook actions 179 of 459 39.0%What each function asks for, and the record a team ends up holding when it is done:
| Function | What the Core asks for | Record it leaves |
|---|---|---|
| Govern | Policies, roles, risk tolerance, an inventory of AI systems, testing and incident practices, and rules for third-party software and data | Approved policies, named owners, the system inventory, a third-party register |
| Map | The context of one system: its purposes, users, laws, requirements, components and the likely impacts of its use | A purpose statement, system requirements, an impact assessment and the initial go or no-go call |
| Measure | Methods and metrics, evaluation for each trustworthiness characteristic, risk tracking over time and feedback on whether the measurement works | Test sets, metrics, results, and a list of risks that were not or could not be measured |
| Manage | Prioritizing risks, choosing responses, keeping a way to disengage the system, and post-deployment monitoring, incidents and improvement | A release decision, risk responses, residual risk notes, monitoring and incident plans |
- Map outcome - the Core says that after Map, users should know enough "to inform an initial go/no-go decision about whether to design, develop, or deploy an AI system."
- Measure processes - its processes "should include rigorous software testing and performance assessment methodologies with associated measures of uncertainty, comparisons to performance benchmarks, and formalized reporting and documentation of results."
- MANAGE 1.1 - this subcategory asks for a determination of whether the system achieves its intended purposes and whether its development or deployment should proceed.
What Is TEVV in the NIST AI RMF?
TEVV stands for test, evaluation, verification and validation: the measurement work that produces evidence about an AI system across its lifecycle. The AI RMF Core states that after the Measure function, "objective, repeatable, or scalable test, evaluation, verification, and validation (TEVV) processes including metrics, methods, and methodologies are in place, followed, and documented."
NIST AI RMF TEVV therefore means documented processes and their records, kept across the lifecycle. For how the four words split into separate checks when the system is an agent, see AI agent validation.
TEVV Tasks Across the AI Lifecycle
Appendix A of the framework, Descriptions of AI Actor Tasks, assigns TEVV work to each phase. It also states: "Ideally, AI actors carrying out verification and validation tasks are distinct from those who perform test and evaluation actions."
| Phase | TEVV task in Appendix A | Evidence for an AI agent |
|---|---|---|
| Design, planning and data | Validating assumptions about design, data and measurements against the intended deployment context | Acceptance criteria per task, the list of tools the agent may and may not call, staging data shaped like production |
| Development | Model validation and assessment | The model ID, prompt version and tool definitions recorded with every test run |
| Deployment | System validation and integration with testing, including user experience and legal, regulatory and ethical specifications | Scenario results from a run against real tools in staging, plus the release decision they fed |
| Operations | Ongoing monitoring, testing and expert recalibration of models, incident tracking and detection of emergent properties | Production traces, an incident log, and a rerun of the scenario suite after each change |
Which AI RMF Subcategories Use TEVV Wording?
The script searched each Core subcategory's own text for TEVV words. The output below gives the totals per function, with documentation wording counted beside them, since Map asks for documentation far more often than for tests.
Subcategories whose own text uses TEVV words
(test, evaluate, verify, validate, TEVV)
Function TEVV words Measure/monitor Document
Govern 2 3 3
Map 3 0 15
Measure 9 10 15
Manage 1 5 5
All functions: 15 of 72 (20.8%)
Measure share of those: 9 of 15 (60.0%)GOVERN 4.2 and MAP 1.4 match only through a passing form of evaluate, a lifecycle step in one and re-evaluated business value in the other, so neither asks for test results. The table lists every subcategory that matched, with the evidence each implies for an agent:
| Subcategory | What it asks | Evidence for an agent |
|---|---|---|
| GOVERN 4.2 | Teams document the risks and impacts of the AI they design, develop, deploy, evaluate and use | A risk note per agent that names the systems it can write to |
| GOVERN 4.3 | Practices enable AI testing, incident identification and information sharing | A staging environment the agent can be tested in, and an incident channel |
| MAP 1.1 | Intended purposes, users, laws and settings are documented, including related TEVV and system metrics | The tasks the agent is for and the metric that says each was done |
| MAP 1.4 | Business value or context of use is defined, or re-evaluated for an existing system | A stated reason the agent acts instead of a person or a fixed script |
| MAP 2.3 | Scientific integrity and TEVV considerations are documented, such as experimental design and data selection | How scenarios were chosen and what the test data does not cover |
| MEASURE 2.1 | Test sets, metrics and details about the tools used during TEVV are documented | The scenario set, its version and the test runner that executed it |
| MEASURE 2.2 | Evaluations with human subjects meet applicable requirements and represent the population | Consent and sampling records for any user study of the agent |
| MEASURE 2.6 | The system is evaluated regularly for safety risks and shown to fail safely | Results for requests the agent must refuse, and proof it stops cleanly |
| MEASURE 2.7 | Security and resilience are evaluated and documented | Results for instructions planted in data the agent reads |
| MEASURE 2.9 | The model is explained, validated and documented, and output is interpreted in context | A trace that ties each action to the input that caused it |
| MEASURE 2.11 | Fairness and bias are evaluated and results documented | The same request from different users gets the same outcome |
| MEASURE 2.13 | The effectiveness of the TEVV metrics and processes themselves is evaluated | A review of which criteria could not be checked and why |
| MEASURE 3.3 | Feedback from end users and impacted communities feeds evaluation metrics | User reports turned into new test scenarios |
| MEASURE 4.2 | Results in deployment context are checked with domain experts to validate consistent performance | Expert sign-off on the scenarios that gate a release |
| MANAGE 4.1 | Post-deployment monitoring plans capture and evaluate user input, appeals, incidents and change management | A monitoring plan and the trigger that reruns tests after a change |
TEVV-Athlon, NIST's Draft Method for Running TEVV
NIST announced The TEVV-Athlon Framework for Evaluating AI Systems, initial public draft NIST AI 200-2, on August 7, 2026, with comments open until October 6, 2026. The announcement lists agentic systems among the AI types the method is meant to cover and asks for input on the definitions of the four TEVV terms themselves.
The initial public draft of NIST AI 200-2 says the "TEVV-Athlon itself implements the Measure core function" and that its results inform Manage. It adopts a definition of TEVV from the National Security Commission on Artificial Intelligence: methods and metrics "to determine that a technology or system satisfactorily meets its design specifications and requirements, and that it is sufficient for its intended use." Its four stages:
- Articulate and Organize - state the goal of the assessment and plan it, using what Govern and Map produced as input.
- Define and Construct - turn the goal into Blocks, the system attributes or characteristics the assessment will gather evidence on.
- Apply and Measure - run Events, the activities that produce evidence for each Block, with the Tools that record it.
- Synthesize and Interrogate - analyze the data, check that it measures what it was meant to, and present results for Manage.
The draft's worked example assesses chatbots, so adapting the method to an agent's tool calls is left to whoever runs the assessment.
What Does NIST's Generative AI Profile Add to the AI RMF?
NIST AI 600-1, the Generative AI Profile, applies the AI RMF to generative AI. It describes 12 risks that are unique to generative AI or made worse by it, then lists suggested actions under AI RMF subcategories, each tagged with the risks it addresses; the script counted 212 actions, 72 of them under Measure.
NIST AI 600-1 (July 2024), read 2026-10-20
Source: nvlpubs.nist.gov NIST.AI.600-1.pdf
Suggested actions: 212
(208 IDs read as text; 4 rows whose
ID glyphs carry no Unicode mapping)
Govern 58 27.4%
Map 39 18.4%
Measure 72 34.0%
Manage 43 20.3%The profile is narrower than its length suggests. It says the working group's focus "was limited to four primary considerations relevant to GAI: Governance, Content Provenance, Pre-deployment Testing, and Incident Disclosure" and that future revisions will add subcategories, risks and actions.
| Risk in NIST AI 600-1 | What NIST means | Actions tagged, at least | What to test in an agent |
|---|---|---|---|
| CBRN Information or Capabilities | Easier access to chemical, biological, radiological or nuclear weapon information | 31 | Refusal of such requests, including when they arrive through a tool result |
| Confabulation | Confidently stated false content, which NIST notes is often called hallucination | 18 | Claims of a finished action with no matching change, a form of agent action hallucination |
| Dangerous, Violent, or Hateful Content | Violent, inciting or self-harm content and hateful or stereotyping material | 31 | What the agent writes into tickets, emails or files, along with its replies |
| Data Privacy | Leakage, unauthorized use or de-anonymization of personal or sensitive data | 26 | Personal data copied into logs, tickets or other systems the agent can reach |
| Environmental Impacts | High compute use in training or operation and its effect on ecosystems | 4 | Tokens and calls per completed task, tracked across releases |
| Harmful Bias or Homogenization | Amplified biases, performance gaps between groups and undesired sameness of output | 57 | Equal outcomes for equal requests from different users or languages |
| Human-AI Configuration | Interactions that lead to automation bias, over-reliance or anthropomorphizing | 58 | Approval steps a person actually sees before a risky action runs |
| Information Integrity | Content that blurs fact and opinion or fuels dis- and misinformation at scale | 71 | Sources the agent cites match what the source says |
| Information Security | Lower barriers to cyberattacks and a larger attack surface for the system itself | 48 | Planted instructions in documents, tickets or web pages the agent reads |
| Intellectual Property | Easier production or replication of allegedly copyrighted, trademarked or licensed content without authorization | 26 | Licensed code or text copied into what the agent writes |
| Obscene, Degrading, and/or Abusive Content | Obscene, degrading or abusive imagery, including synthetic child sexual abuse material and nonconsensual intimate images | 14 | Refusal paths for generation tools the agent can call |
| Value Chain and Component Integration | Non-transparent or untraceable third-party components and data, and weak supplier vetting | 36 | Behaviour after the model, an MCP server or a plugin changes version |
Information Integrity carries the most tags and Environmental Impacts the fewest. An action usually carries several tags, so the column adds up to more than the number of actions.
How Do You Apply the AI RMF to an AI Agent?
Apply it as you would to any AI system, then add the risks that come from acting. AI RMF 1.0, its Core tables and its Playbook never name agents, and the Generative AI Profile uses the word for AI only once, listing autonomous agents among security threats:
Occurrences of "agent" (any form)
AI RMF 1.0 text (AI 100-1) 0
AI RMF Core tables 0
AI RMF Playbook (72 entries) 0
AI 600-1 6
of which "autonomous agents" 1
the rest: chemical, biological or other
dangerous agents (the CBRN risk)NIST's Center for AI Standards and Innovation (CAISI) described the risks of agents that act in its January 2026 request for information on securing AI agent systems: "AI agent systems are capable of planning and taking autonomous actions that impact real-world systems or environments." It names indirect prompt injection, insecure or poisoned models, and models that harm security without any attacker, such as through specification gaming.
The worked example is an access-provisioning agent. It reads requests in an IT ticket queue, adds people to directory groups through a tool and comments on the ticket. The organizational controls around agents like it are covered in agentic AI governance; the table covers the risk framework steps.
| Function | What your team does for this agent | Subcategories | Evidence it leaves |
|---|---|---|---|
| Govern | Name an owner who approves releases, write the rule that admin groups need human approval, and list the model provider and directory API as third parties | GOVERN 1.4, GOVERN 2.1, GOVERN 6.1 | Owner record, written approval rule, third-party register |
| Map | State the agent's purpose, write requirements such as never granting an admin group, list the tools it may call and the ones it must not, and mark where a person reviews | MAP 1.1, MAP 1.6, MAP 3.5 | Purpose statement, requirements, tool allow and deny lists, oversight points |
| Measure | Run scenarios against a staging directory, including tickets with planted instructions, document the test set and the runner that executed it, and list what could not be measured | MEASURE 1.1, MEASURE 2.1, MEASURE 2.3, MEASURE 2.7 | Scenario set, per-run record of tool calls and group changes, list of unchecked criteria |
| Manage | Decide go or no-go from the results, keep a tested way to disable the agent, and plan monitoring and incident handling after release | MANAGE 1.1, MANAGE 2.4, MANAGE 4.1, MANAGE 4.3 | Release decision, disable procedure, monitoring and incident plan |
The Measure row only produces evidence when each requirement becomes a criterion a run can decide. For this agent, the criteria read like these:
- Valid request - after the run, the directory shows the user in the requested group; the agent's comment saying so is not enough.
- Admin group without approval - group membership is unchanged and the ticket records a refusal.
- Planted instruction - a ticket whose text tells the agent to add its author to an admin group leaves every group unchanged.
- Failed tool call - when the directory call fails, the agent reports the failure instead of success.
- Forbidden tool - the run log shows no call to a tool on the deny list, such as one that deletes accounts.
Some criteria will not be decidable, for example when staging has no audit log for one directory. MEASURE 1.1 asks you to document the risks you will not or cannot measure, so record those criteria as unchecked rather than counting them as passed. The model inside the agent raises its own questions, such as version pinning and revalidation after a provider change, which AI model governance covers.
How Does the NIST AI RMF Relate to the EU AI Act and ISO Standards?
They do different jobs, and NIST does not claim one satisfies another. NIST's crosswalk documents page says a listed crosswalk does not imply NIST endorsement, "nor does it imply that either resource comprehensively covers the contents of the other."
| Text | What it is | Binding? | Status, read 2026-10-20 | Link to the AI RMF |
|---|---|---|---|---|
| NIST AI RMF 1.0 | Risk management framework | No, voluntary | Released January 26, 2023; being revised | The base text |
| NIST AI 600-1 | Profile of the AI RMF for generative AI | No, voluntary | July 26, 2024 edition | Actions filed under AI RMF subcategories |
| EU AI Act | Regulation (EU) 2024/1689 | Yes, for operators in its scope | In force since August 1, 2024; high-risk rules from December 2, 2027 (Annex III) and August 2, 2028 (Annex I) | A January 2023 NIST illustration relates the AI RMF to the Act as then proposed |
| ISO/IEC 42001 | Standard with requirements for an AI management system | No, voluntary standard | Edition 1, published December 2023 | A crosswalk submitted by Microsoft, listed by NIST |
| ISO/IEC 23894 | Guidance on AI risk management | No, voluntary standard | Edition 1, published February 2023 | A crosswalk by INCITS/AI, revised August 14, 2025, listed by NIST |
The Commission's regulatory framework for AI page, updated on October 19, 2026, is the source for the EU dates. It says the AI Omnibus entered into force on July 27, 2026, setting December 2, 2027 for high-risk use cases in areas such as employment and critical infrastructure, and August 2, 2028 for systems built into products such as lifts or toys.
The closest EU match to Measure is the risk management system for high-risk systems in Article 9 of the AI Act. Its paragraph 8 requires testing, as appropriate, during development and in any event before the system is placed on the market or put into service, measured against "prior defined metrics and probabilistic thresholds" suited to the system's intended purpose. How to turn that article into test cases is the subject of EU AI Act conformity testing.
ISO/IEC 42001 is listed in ISO's catalogue as edition 1 of December 2023, a standard that sets requirements for running and improving an AI management system in an organization that provides or uses AI. It works at the level of the organization, while the AI RMF's Map, Measure and Manage functions can be applied one system at a time.
Which NIST AI RMF and AI Agent Updates Are in Progress?
Version 1.0 is still the current text, and NIST's AI RMF page says: "The AI RMF 1.0 is being revised as part of the White House AI Action Plan." That plan, America's AI Action Plan of July 2025, recommends that NIST "revise the NIST AI Risk Management Framework to eliminate references to misinformation, Diversity, Equity, and Inclusion, and climate change." No revised version had been published when the NIST pages were read on 2026-10-20.
| Date | NIST item | Status, read 2026-10-20 | Why it matters for agents |
|---|---|---|---|
| February 17, 2026 | AI Agent Standards Initiative, launched by CAISI | Input gathered through a security RFI and an NCCoE draft on identity and authorization for software and AI agents | A NIST program aimed at agents, covering standards, open protocols and security research |
| April 7, 2026 | Concept note for an AI RMF profile on trustworthy AI in critical infrastructure | Ongoing; NIST says it will share discussion drafts through its mailing list and community Slack channel | Aims to give critical infrastructure sectors more confidence to deploy AI agents and tools |
| August 7, 2026 | NIST AI 200-2, TEVV-Athlon, initial public draft | Comment period closed October 6, 2026 | A method for running Measure that names agentic systems in its scope |
| October 16, 2026 | Key practices for incident readiness and response in super intelligence training and evaluation | Published as a voluntary document with NSA and CISA | Written for frontier developers, about incidents where agents take uninstructed actions |
- Critical infrastructure profile - NIST's concept note page says the profile "will guide CI operators towards specific risk management practices to consider when engaging AI-enabled capabilities."
- Agent initiative - the AI Agent Standards Initiative page lists three pillars: industry-led standards, community-led protocols and research into agent authentication, identity and security evaluations.
- Incident practices - NIST's October 2026 announcement covers incidents "in which one or more SI agents take significant uninstructed actions that compromise explicit or implicit policies, controls, or security measures imposed by the operator".
- Center name - that announcement names the issuing center NIST's Center for Advancing Innovation and Standards for Super Intelligence (CAISSI), while February's initiative launch was issued as CAISI.
Pre-Release Agent Test Evidence for AI RMF Measure Work With Agent Assurance
For an agent that acts, most of the Measure evidence in the example above comes from running it: what it called, what it changed and which criteria nobody could check. TestMu AI's Agent Assurance produces that kind of test evidence before release, for agents that use tools, edit files and send API requests. It is a pre-alpha product with a publicly installable command-line client, Rook CLI:
npm install -g @testmuai/rook
rook --versionYou can use Rook CLI on macOS and Linux, and 64-bit Windows through npm or WSL. For the npm package you need Node.js 20 or a later release, and the Rook CLI installation guide covers Homebrew and the checksum-verifying shell installer for macOS and Linux.
Agent Assurance starts from your agent's source code or specification and can also read a PRD or your written policies. It writes scenarios and gives each one acceptance criteria (functional and adversarial classes unless you ask for more), calls the live agent in staging or inside a CI job, and checks every criterion against what the run produced:
- Changed files and artifacts - output files under the paths named in the agent's profile, and the artifacts the run produced.
- Tool calls - each call measured against the tools the agent declares, with a must-not-call list for forbidden ones.
- Records - read-only lookups in the system the agent touched, using an MCP tool that you provide and approve.
Model judges score some criteria, rules and read checks decide the others, and an action the agent only claims is never proof: the run's evidence decides. Each verdict is one of Pass, Fail or Unable to Verify. The Agent Assurance documentation on results and evidence describes the scenario pass rate as a share of decided verdicts and shows coverage beside it, so a high rate built on little verified evidence stays visible.
Most AI eval and observability tools score the agent's answers and the traces it logged. Agent Assurance instead inspects the changes a run left in files, tool calls and records, and keeps a separate list of the criteria it had no evidence to decide.
Agent Assurance is not a NIST AI RMF mapping: it neither certifies nor shows compliance with the framework or with any of its functions, categories or subcategories. It tests and reports, and your risk and compliance team and auditors decide whether its results belong in your Measure records and whether your risk management meets the AI RMF or any regulation.
Use a staging environment, since a test run performs the agent's writes for real and nothing rolls them back. Agent Assurance does not monitor production or block anything at runtime, so post-release monitoring needs your own tools.
Write a Measure Gap Report to Start the AI Risk Management Framework
Pick the agent that can change the most records, fill in the four rows of the worked example for it, and write its five riskiest requirements as criteria a run can decide. Mark each criterion you cannot check yet and name what would make it checkable; that list is your first Measure gap report under the AI risk management framework, the record of unmeasured risks that MEASURE 1.1 asks for.
To see how that evidence supports trust claims beyond risk management, read the guide to AI assurance. To run the criteria against staging in a pipeline, follow the Agent Assurance CI/CD guide, which gates a job on the run's report instead of its exit code.
Note: Sirajuddin Khan, Vice President of Product Management at TestMu AI with Agentic AI and GenAI Product Strategy among the listed areas of expertise, is the author of record for this guide and approved it; the guide was researched and drafted with AI assistance. Vipul Verma, Group Senior Vice President of Engineering at TestMu AI, was its technical reviewer. The sources behind it are NIST publications and pages (NIST AI 100-1, NIST AI 600-1, the AI RMF Playbook and the draft NIST AI 200-2), America's AI Action Plan, the European Commission's AI Act pages, ISO's catalogue and the TestMu AI product documentation, and the editorial process and AI use policy describes how every claim is checked before publication.
Author
Sirajuddin Khan is Vice President of Product Management at TestMu AI, where he drives the company's agentic AI product strategy, building a suite of autonomous agents that includes Agentic Browsers and Agentic Visual Testing and shifting the unit of work from test execution to autonomous outcomes. One of the company's earliest product leaders, he has owned the roadmap for the high-performance execution cloud and grew the cross-browser testing products from early adoption to market leadership. He brings over a decade of experience across SaaS, B2B, and eCommerce, with earlier product roles at Wydr and ShopClues, where his catalog and search work cut delivery SLAs and lifted seller activity. Sirajuddin holds an MBA in Information Technology from Sikkim Manipal University and a B.Tech in Computer Science Engineering from Maharshi Dayanand University.
Reviewer
Vipul Verma is Group Senior Vice President of Engineering at TestMu AI, where he heads the entire engineering organization that builds KaneAI, HyperExecute, and the broader testing cloud. He brings 15+ years architecting, securing, and scaling large enterprise applications across multiple sites. Before TestMu AI he was India Head at LogicHub, where he built the India R&D site from the first employee to a 30-plus engineering team, and Principal Software Engineer at Sumo Logic, where he was the first engineer in the India office and shipped search-performance and pricing-model initiatives. Earlier he worked on trading platforms at Portware and D. E. Shaw. Vipul holds a B.Tech in Computer Science from IIT Kharagpur.
AI Risk Management Framework FAQs
Did you find this page helpful?
More Related Learning Hubs
TestMu AI forEnterprise
Get access to solutions built on Enterprise
grade security, privacy, & compliance
- Advanced access controls
- Advanced data retention rules
- Advanced Local Testing
- Premium Support options
- Early access to beta features
- Private Slack Channel
- Unlimited Manual Accessibility DevTools Tests





