AI Agent Assurance for Your Agent Deployments

Agent Assurance Tests How Your Agent Actually Behaves across Workflows, Tools, and Actions to Catch Failures and Vulnerabilities.

npm install -g @testmuai/rook

Thought LeadershipAI

Agent Autonomy Should Be Earned, Not Granted

Agent autonomy should be earned with evidence, not granted by default. Use the Earned Autonomy Ladder to promote, and demote, AI agents safely.

Published on:

Here's the paradox that CIOs are facing this quarter. Genpact did a survey of over 2000 execs and published the results in November 2026. 79% said they have AI agents in production, but humans still decide or audit in at least 87% of cases (Genpact, 2026).

So the agents are definitely running, but humans are still reviewing (auditing) nearly all of it. That's where the ROI is currently getting stuck.

My argument is that the current limit on agent autonomy isn't what the models can do. It's the lack of evidence. You can't give an agent any more authority because you can't prove it's earned it.

Authority given without that proof is in danger of being taken away in the public eye, which is precisely the pattern that Gartner's "demotion forecast" describes. This article sets out a model for doing it properly: the Earned Autonomy Ladder.

TL;DR

Agent autonomy is a permission that the company deploying the agent grants on a workflow-by-workflow basis. It should never be the default. An agent should progress from observing to advising to acting with approval to acting alone only when there is independent evidence that it is ready to do so.

There should be demotion triggers for each level too, documented prior to anything going wrong. Most enterprises have agents in production. Very few have the evidence that would justify giving them any more autonomy.

How Does an AI Agent Earn More Autonomy?

  • AI agent autonomy levels: Gartner's four AI agent autonomy levels are Observe, Advise, Act with Approval and Act Autonomously. The Earned Autonomy Ladder uses them as rungs and sets the level per workflow, which is bounded autonomy: one agent can act alone on password resets and only advise on refunds.
  • Promotion Packet: The Promotion Packet is the evidence an agent needs to move up a rung: sealed test-suite results, adversarial and failure-mode tests, permission-boundary tests on the agent harness, a pause drill that held and an audit that does not use the model under review.
  • Demotion triggers: Demotion triggers are written before go-live and drop the agent one rung while the issue is fixed: a permission-boundary breach, a failed pause drill, drift beyond a threshold or a vendor model change that has not been re-verified.

Where Do Human in the Loop AI Agents, AI Evals and Monitoring Fit?

  • Human in the loop AI agents: Human review is the right control at the Advise and Act with Approval autonomy levels. The Earned Autonomy Ladder adds a written level of evidence that lets review shrink for one workflow, instead of staying permanent and the same for every agent.
  • AI evals: Offline AI evals on a sealed, versioned suite count as promotion evidence only when the grader is not the model doing the work: deterministic assertions, a different model or both.
  • AI agent monitoring: AI agent monitoring and agent observability show what an agent did after it acted, which makes them the demotion signal. They cannot show, before release, that an agent will stay inside its permissions.
  • AI assurance: Pre-production AI assurance is the promotion signal. TestMu AI's Agent Assurance runs generated scenarios against the real agent before release, checks what each run changed rather than what the agent reported, and its report can gate a release through Agent Assurance CI/CD.

How Does the Ladder Fit Existing Governance?

  • Agent inventory: Gartner's steps for agent sprawl include a centralized agent inventory built with AI TRiSM tools. Adding an autonomy level field to each entry tells the governance team how much evidence that agent needs.
  • Change management: A promotion reviewed like a production release, with its evidence pack attached, puts the Earned Autonomy Ladder inside the ADLC release gates rather than a crisis meeting.
  • Audit trail: Tamper-proof, human-readable Promotion Packet evidence answers an AI audit and raises AI verifiability: the share of the agent's behavior that someone other than the agent can confirm from evidence.

Why Are AI Agents in Production yet Still Under Supervision?

The evidence that would justify removing the human simply doesn't exist. "In production" therefore usually just means "in production with a person checking".

The numbers from three separate studies point the same way:

  • Genpact (Nov 2026): 79% have agents in production, only 15% have AI investment fully tied to P&L (Genpact).
  • Dynatrace State of Observability 2026 (Nov 2026): humans are verifying, on average, 66% of AI-powered decisions. Just 27% of organizations are looking at the visibility of how the agents make their decisions and take their actions (Dynatrace).
  • G2 2026 AI Agents report (Oct 2026): 52% require human quality review prior to go-live and 77% re-assess agent performance at least on a monthly basis (G2).
Survey data showing most enterprises run AI agents in production while humans still review most agent decisions.

Each survey used its own sample and its own questions, so the figures shouldn't be added together. The direction is still the same, though. Agents are out there; people are still checking a large proportion of their work, and very few organizations can see enough of an agent's decision-making to stop checking.

Human review isn't the issue per se. It's the appropriate control for many workflows.

The issue is that review is generally permanent and the same for all. No one has defined what level of evidence would allow review to be reduced in a particular workflow, so it never is. The cost of the agent remains pretty much the same as the cost of the human it was intended to assist.

What Are the AI Agent Autonomy Levels?

Gartner defines four levels: Observe, Advise, Act with Approval, and Act Autonomously. Gartner cautions that if you apply the same level of governance to every agent regardless of its level, you're going to end up with agent failure (Gartner, May 2026).

The same Gartner release makes a prediction that every CIO should take note of. By 2027, 40% of enterprises will be downgrading or decommissioning autonomous AI agents due to governance gaps.[1] Gartner's reason for this is governance gaps, not model capability.

I agree with the diagnosis and would go even further. A demotion in itself is not a failure. The failure is a demotion that was completely unanticipated, brought about by an incident and decided in a crisis meeting.

A demotion that was planned is simply a control doing its job. The difference between the two is whether the conditions for promotion and demotion were in place prior to the agent going live.

The scale makes this urgent. In April 2026, Gartner said only 13% of organizations think they have the right AI agent governance in place. It also projected that an average Fortune 500 enterprise will have more than 150,000 agents in use by 2028 (Gartner, Apr 2026).

At that number, a meeting cannot decide the autonomy level of each agent. The decision has to be backed by evidence and recorded in the agent inventory.

The Earned Autonomy Ladder: A Framework for Agentic AI Governance

The Earned Autonomy Ladder is a governance model in which an AI agent's level of autonomy is a revocable permission. The company granting the autonomy gives it on a per-workflow basis only after independent evidence has cleared a written promotion threshold. Demotion triggers are all defined in advance.

The rungs are Gartner's four levels, so we're using the language that your analysts and your auditors are already familiar with. What the ladder adds is the stuff in between the rungs: the promotion evidence, who signs off and what causes the agent to be sent back down.

Earned Autonomy Ladder showing four AI agent autonomy levels with promotion evidence and demotion triggers.
RungWhat the agent may doPromotion evidenceWho signs offDemotion trigger
1. ObserveRead-only; outputs seen by its ownerInventory entry: owner, scope, data access, model(s) usedAgent ownerAccesses data outside declared scope
2. AdviseRecommends; a human executes every actionPass rate on a sealed, versioned test suite for this workflow; tamper-proof logs of every recommendationBusiness owner + QAPass rate drops below threshold on any regression run, including after a model or prompt change
3. Act with ApprovalExecutes after human approval per action or batchRung-2 evidence plus adversarial and failure-mode tests (prompt injection, tool failure, bad data); permission-boundary tests proving it cannot exceed granted authority; working pause controlRisk + security + business ownerAny permission-boundary breach; approval-override rate above an agreed level; failed pause drill
4. Act AutonomouslyExecutes within a bounded scope; humans audit samplesRung-3 evidence plus sustained clean record at rung 3 over an agreed period; independent audit that does not use the model under review; incident disclosure path rehearsedExecutive accountable ownerAny severity-1 incident; drift beyond threshold in continuous testing; vendor model change not yet re-verified

You can read the table in three ways.

  • Autonomy is set per workflow rather than per agent. An agent can be set to 'Autonomous' when it comes to resetting passwords and at the same time be set to 'Advise' when it comes to refunds. That's what bounded autonomy looks like in practice.
  • Every promotion has its evidence. If the evidence isn't there, then the agent doesn't get the promotion.
  • Demotion is automatic and entirely expected. If a trigger fires, the agent just drops a rung as the issue gets resolved. That's the control working; it's not an embarrassment in the slightest.

What Evidence Should an Agent Have Before It Acts Autonomously?

The agent needs a Promotion Packet. That's a bundle of independently produced, tamper-proof evidence showing what the agent does on known cases, on hostile cases, and when it hits its permission limits. It has to be stored in the deploying company's own systems.

The Promotion Packet is where TestMu AI's principles for AI trust become tests that you can actually run. Here's the checklist we'd apply:

  • Model diversity: There is no single model that is the sole dependency for the outcome, and the model that does the work is never the one that grades it. If the agent is running on Model A, the verdict comes from the deterministic assertions, a different model or both.
  • Evidence: Every meaningful action in the test run leaves a record that is tamper-proof and readable by a person: inputs, tool calls, outputs and verdicts.
  • Continuous system testing: the suite runs again on every model, prompt, tool or data change (including the failure and attack cases). A single pass is not sufficient.
  • Deployer-defined authority: the agent is unable to access data or actions outside of the scope that the company has granted it, even when instructed to do so in a prompt.
  • Independent audit: the audit trail and the scoring are completely independent of the model under review and of the vendor whose model it is.
  • Containment: A named individual has paused the agent in the middle of a task during a drill, and the pause has been maintained.
  • Incident disclosure: There is a written path for telling affected parties, and it has been rehearsed.

Two practical points on top of the checklist.

Continuity: if the vendor can pull the model or quietly change it, then your evidence is only as current as your last re-verification. That's why 'vendor model change not yet re-verified' is a demotion trigger.

Location: the verification has to run where the model runs, and the evidence has to end up in the customer's own control plane, not in the vendor's dashboard.

Regulators are heading in the direction of the containment requirement. For high-risk systems, Article 14 of the EU AI Act requires that the people who are in control of the system be able to 'stop' it via a 'stop' button or similar procedure (EUR-Lex, Regulation (EU) 2024/1689).

Not every agent is a high-risk system under the Act. A pause drill is cheap, though, and it's the clearest rung 3 test you can have.

It also fits neatly with the Govern, Map, Measure and Manage functions of the NIST AI RMF. The Promotion Packet is your Measure evidence, and the demotion triggers are your Manage controls.

Is AI Agent Monitoring Sufficient to Grant Autonomy?

No. Monitoring and agent observability will tell you what the agent did once it's done. Promotion evidence will tell you what it will do prior to you allowing it to.

This is where the market is moving the fastest. On 3rd November 2026, CrowdStrike announced it had entered into a definitive agreement to acquire Fiddler for agent observability. Their CEO described Fiddler as "the flight recorder for every agent" (CrowdStrike).

The WSJ reported the price as being around $400m (PYMNTS).

Gartner expects spending on AI governance platforms to be $492m in 2026 and to cross the $1bn mark by 2030 (Gartner, Feb 2026).

Flight recorders are of immense value. A flight recorder is not in itself a certificate to fly, though.

Aviation uses both, and they answer different questions. The survey data shows that the companies have built the recorder but not the certification.

  • LangChain's State of Agent Engineering survey found that nearly 89% of respondents have observability for their agents; only 52.4% run offline evaluations. Quality was the top barrier to production (32% - LangChain).
  • KPMG's Q3 2026 AI Pulse shows that 62% of organizations are either building, deploying or developing agents. The proportion building in the areas of controls, monitoring and evaluation has dropped to 30% from 43% two quarters previously (KPMG).
QuestionAgent observabilityPromotion evidence (assurance)
What did the agent do yesterday?✅ Traces, scores, dashboards-
Should this agent be promoted to the next rung?Partial: lagging indicator only✅ Sealed suites, adversarial cases, boundary tests
Will it stay inside its permissions under attack?Detects breaches after the fact✅ Proves it before release
Did the vendor's model update change behavior?Shows drift once users hit it✅ Re-verifies before the change reaches users
Who owns the evidence?Often the vendor's platformShould be the deployer's control plane
Comparison of agent observability and Agent Assurance in an agentic AI governance program.

The two halves absolutely need each other. Observability provides the demotion signal. Assurance provides the promotion signal.

A governance program with just one of those is only ever going to work in one direction.

Edge Cases That Break Naive Autonomy Models

The ladder has to be able to cope with the cases that break the simpler models. Here are four.

  • We built the agent in-house so we can validate it ourselves. You can and you should. Independence in this context is independence from the model rather than from the company. An in-house team that validates its agent against itself using deterministic checks and a different model is fine. An in-house team that has its agent validate itself against its own model is not.
  • Our agent is low risk, so this is overkill. Keep it at rung 1 or 2 where the evidence required is minimal. The whole point of the ladder is to make light governance easy to justify in the case of low-risk workflows and to require heavy governance only when the agent is acting on its own.
  • 'The evidence lives in our vendor's platform' (it's then their evidence). If you change vendors or the vendor changes their terms, your audit trail goes with them. Promotion evidence should be exportable and should reside in your control plane.
  • Demotion will kill adoption. I think it actually does the opposite. If you know there's a trigger that will automatically pull the agent back, then teams are much more willing to promote. Without that safety net, the safe thing to do is just to have humans checking everything forever. That's exactly the supervision levels we see in the Genpact data.

What This Means for the Testing Market

Testing used to be about one thing: does the software work? Agentic AI governance raises a second question: has this agent earned the right to act? I think the second question is going to be the bigger market, as it sits between AI spend and AI return.

I expect software testing to become a $65-100bn market over the next 10 years. That's my estimate, not a prediction from some research firm. It's based on this shift: the budgets that currently pay people to review agent output will move to budgets that pay for the generation of evidence, which in turn will allow the review stage to get progressively smaller.

My predictions:

  • By 2027, 'autonomy level' will be a standard field in the enterprise agent inventory (alongside owner and data scope). Gartner already recommends that you build an agent inventory as part of your agent sprawl management strategy. Autonomy level is the field that tells you how much governance each agent entry requires.
  • Autonomy promotions will move into change management and will be reviewed as production releases with an evidence pack attached (that's the existing approval process that the auditors already trust).
  • Observability and assurance vendors will all be converging on the scoring layer. The distinction will be who produces the evidence prior to release and who owns it.

How TestMu AI Approaches Agent Autonomy

At TestMu AI, we build the independent verification layer for this ladder. Our view is that verification should generate evidence rather than just opinions.

  • Human<->Software: KaneAI and Kane CLI generate and run deterministic tests for web and mobile apps, on Real Device Cloud and at scale via HyperExecute.
  • Human<->Agent: Agent Testing tests chat, voice, IVR and messaging agents prior to them going to customers.
  • Agent<->Agent: Agent Assurance, delivered by Rook, has agents test agents with no human handshake. This is the pre-production evidence a rung-3 or rung-4 promotion needs.
Agent Assurance run of 6 scenarios showing 3 Passed, 0 Failed and 3 Unable to Verify, above the SC-004 result for a T-1041 refund and plan-upgrade request, where all 7 acceptance criteria pass and the expanded C3 shows the customer reply only logging the ticket as S2 and routing it to billing.

We're on the same ladder. Our judgments come from a layer separate from the model under review, our tests are deterministic, and the evidence is constructed to sit in the customer's control plane rather than remain with us.

Start a 14-day free trial of Agent Assurance by TestMu AI, with 4,000 credits, to test how your AI agents behave across workflows, tools, and actions

Conclusion: Make Agent Autonomy a Decision Backed by Evidence

The industry has spent two years working out how capable agents are. The question that ultimately determines ROI is whether an agent has in fact earned more autonomy inside a given workflow and whether you can prove that.

Autonomy that is simply granted by default is the kind that gets revoked in a crisis. The kind of autonomy that is earned through independent evidence and which is then governed by pre-defined demotion triggers is the kind that sticks.

Start small. Pick one agent workflow that people are still looking at today. Note its current rung, the evidence that would take it up a rung, and the evidence that would take it down a rung.

That one page is your first Promotion Packet.

Shipping agents that other agents use? Rook by TestMu AI does Agent Assurance: agents testing agents before release, with no human in the loop.

To run your first test, install Agent Assurance with npm install -g @testmuai/rook@latest command, or
for a full walkthrough of the product.

Author

...

Mayank Bhola

Blogs: 8

  • Twitter
  • Linkedin

Mayank Bhola is Co-Founder and Head of Products at TestMu AI, where he leads the entire product portfolio across KaneAI, Kane CLI, HyperExecute, SmartUI, the Real Device Cloud, Accessibility, and other software testing product lines. As an early Lead Architect he designed and built the company's flagship Tunnel technology from scratch, created the React-based automation platform, and architected the data-intensive pipelines and FAAS services that scale it. He brings more than 10 years of experience in software development and product engineering, with earlier roles as Head of Technology at Juggernaut Books and Senior Software Engineer at PressPlay TV and Zomato. Mayank holds a B.Tech in Computer Engineering from JIIT Noida.

Reviewer

...

Vipul Verma

Reviewer

  • Linkedin

Vipul Verma is Group Senior Vice President of Engineering at TestMu AI, where he heads the entire engineering organization that builds KaneAI, HyperExecute, and the broader testing cloud. He brings 15+ years architecting, securing, and scaling large enterprise applications across multiple sites. Before TestMu AI he was India Head at LogicHub, where he built the India R&D site from the first employee to a 30-plus engineering team, and Principal Software Engineer at Sumo Logic, where he was the first engineer in the India office and shipped search-performance and pricing-model initiatives. Earlier he worked on trading platforms at Portware and D. E. Shaw. Vipul holds a B.Tech in Computer Science from IIT Kharagpur.

Add to Google preferred sources

Summarise with AI

Copied to Clipboard!
...

3000+ Browsers. One Platform.

See exactly how your site performs everywhere.

Try it free
...

Write Tests in Plain English with KaneAI

Create, debug, and evolve tests using natural language.

Try for free

FAQs

Did you find this page helpful?

More Related Blogs

TestMu AI forEnterprise

Get access to solutions built on Enterprise
grade security, privacy, & compliance

  • Advanced access controls
  • Advanced data retention rules
  • Advanced Local Testing
  • Premium Support options
  • Early access to beta features
  • Private Slack Channel
  • Unlimited Manual Accessibility DevTools Tests