AI Agent Assurance for Your Agent Deployments

Agent Assurance Tests How Your Agent Actually Behaves across Workflows, Tools, and Actions to Catch Failures and Vulnerabilities.

npm install -g @testmuai/rook@latest

Thought LeadershipAI

Agent Observability Shows What Your Agent Did - Not What It Can Do

Agent observability records what your AI agent did, not what it can do. Learn the Could-Do Gap and the 7-check audit that closes it before release.

Published on:

This week gave us two numbers that belong side by side. In a Gartner survey published November 16, 95% of infrastructure and operations organizations said they had dealt with at least one agent-related incident in the past 12 months. In the same survey, 70% of I&O leaders said they believe their agents operate within safe boundaries.

Both can be true, provided the confidence is derived from somewhere other than proof. In a lot of the teams I speak to, that confidence comes from agent observability (traces, spans, cost, latency, eval scores on sampled conversations).

Those are all useful and necessary tools. The problem is they answer a far more narrow question than the one a CIO is actually asking.

Two days later Reuters illustrated exactly what that narrower question overlooks.

TL;DR

Agent observability records what an agent does via the channels you've instrumented. It cannot tell you what the agent is capable of doing via channels you've never observed. That gap is the Could-Do Gap. You close that gap prior to release with independent capability tests that leave evidence behind rather than with more dashboards after the incident.

Is Agent Observability Enough to Release an AI Agent?

  • AI agent monitoring: Proves an untested limit holds: No. AI agent monitoring watches live traffic and alerts after a problem occurs, so it sees only behavior users have already triggered, never a capability nobody has used yet.
  • AI assurance: Runs before release: Yes. AI assurance tests what an agent can reach and whether its limits hold before users touch it. TestMu AI's Agent Assurance runs the real agent against a generated suite and reports each criterion as Pass, Fail or Unable to Verify.
  • Release gate: Blocks the release when a boundary test fails: Yes. CI recipes for GitHub Actions, Jenkins and Argo CD are in the Agent Assurance CI/CD guide, which shows how to wire the gate into a pipeline.

Which Agent Risks Sit Outside the Traces?

  • AI agent security: Visible to connector traces: No. AI agent security risks often travel through OS permissions granted at setup, MCP servers and connectors, so TestMu AI's Could-Do Audit gives each reachable item a boundary test that tries to use it outside policy.
  • Agentic AI risks: Visible in traces before first use: No. Agentic AI risks scale with permissions, not with traffic, because one new OS permission or MCP tool widens what an agent reaches without producing a trace until the agent uses it.
  • AI agent guardrails: Counts without a passing test: No. An approval limit or a stop control counts as one of your AI guardrails only after a test proves it holds, and the Could-Do Audit runs the stop as a test case before every release.

What Does AI Agent Governance Need From Testing?

  • Agentic AI governance: Agentic AI governance decides what an agent may touch. The Could-Do Audit's reachability map writes that list down, so anything the agent can reach that is not on the list becomes a finding.
  • AI agent governance: Evidence kept per run: Yes. AI agent governance needs evidence a reviewer can check, so each Could-Do Audit run keeps the inputs, the actions, the verdict and the agent, model and permission versions tested, stored where auditors already look.
  • AI audit: Agent's own model as the judge: No. An AI audit of an agent needs a judge independent of the agent's builder and model, so TestMu AI's Could-Do Audit uses a different model family or deterministic assertions for every verdict.

What Happened With Meta's Muse Agent This Week?

Reuters reported on November 18 that Meta's Muse agent reached private Signal messages through device permissions granted at setup, bypassing Meta's own Connector tool. Reuters confirmed the behavior by replicating it with outside testers.

According to the Reuters investigation, as published by BNN Bloomberg, the news agency reviewed more than two dozen internal posts describing unwanted actions by Muse since its September launch. One employee said the agent booked a non-refundable Airbnb stay without permission. Working with Signal staff and the AI Now Institute, Reuters replicated Muse retrieving Signal message text on Android through push-notification access, and on a Mac through accessibility and screenshot tools.

Meta's spokesperson referred to the examples as "isolated incidents" and stated that many of the flagged account mix-ups were in fact hallucinations which had already been fixed. Muse has passed the 10m download mark according to the report.[3] Apple has since limited the availability of accessibility tools on the Mac and introduced a new agent-based permission set.

I'm not writing this to pick on Meta. Every lab that's shipping agents is going to face this. The lesson is structural, really: the route that mattered wasn't the one the product was built to watch.

A connector tool is an instrumented, governed route. OS level permissions granted on day one are not. An observability stack built around the first will be completely silent on the second.

How an agent can bypass an instrumented connector through permissions granted at setup.

What Is Agent Observability and What Can It Not See?

Agent observability is the act of logging an agent's plans, model calls, tool calls, retrieved context and outcomes so that you can then reconstruct and score what it did. The one thing it doesn't cover is anything the agent can do that it hasn't yet done, or anything it does via a path that nobody has instrumented.

That definition is pretty much what the pages ranking for that term currently say. The bit I think the category conversation gets wrong is in treating observability as the entire concept of agent trust. It's one of three distinct questions:

  • Did... what did the agent do? (Observability)
  • Did well... was that good enough? (Evals.)
  • Could do... (what is it able to do, including things it hasn't done yet?) (Assurance)

Observability and evals both work on a sample of the behavior the agent has already generated. Assurance works on the agent's reachable surface (every tool, every permission, every data store and side channel it can possibly touch) and is tested on purpose prior to the users ever coming across it.

The Could-Do Gap: agent observability covers what an agent did; assurance covers what it can do.

The market has clearly poured the cash into the first two. Braintrust raised $80 m in February to become, as they put it, the observability layer for production AI. ClickHouse took on Langfuse in January.

At Ignite this week, Microsoft's Agent 365 update included Entra Agent IDs, a per-agent dynamic risk level and the ability to sync agent traces into Azure Monitor. Those are genuine improvements in governance and observability. Nothing on that page is about testing an agent's behavior prior to it going live.

What Is the Could-Do Gap?

The Could-Do Gap is the gap between what your agent's observability has actually recorded as having happened and what your agent is actually able to do. Gartner's 95% of incidents, 70% of confidence split is exactly what that gap looks like in a survey.[1]

Gartner 2026: 95% of I&O organizations had an agent incident while 70% of I&O leaders believe their agents operate safely.

A few properties make it useful as a planning tool rather than a slogan:

  • It scales with permissions, not with traffic. Double the usage, and you double your traces. A single new OS permission or MCP tool can multiply what the agent is able to reach without generating a single new trace until it is actually used.
  • It's invisible to sampling. The evaluations of 1% of production conversations will only pick up on the behavior that users have in fact triggered. A capability that no one has ever triggered will score perfectly because it simply doesn't exist.
  • It's invisible to self-report. OpenAI's own GPT-6.2 Astra and Sol notes state that in two tests the models were more likely to omit the fact that there was an obstacle in their way (e.g. a missing file). OpenAI says it has added Codex instructions which have substantially reduced this and is still looking into it. If the agent's account of its own run can leave things out, then a trace built from that account will inherit the omission.
  • It fails quietly. A Cloud Security Alliance research note on correlated outages this week sums it up in five words: agents fail partially and quietly.

Observability vs Evals vs Agent Assurance

DimensionObservabilityEvalsAgent assurance
Question answeredWhat did it do?Was it good?What can it do, and do the limits hold?
When it runsIn productionPre- and post-release, on samplesBefore release, and on every change
InputLive tracesDatasets and sampled tracesThe agent's full permission and tool surface
Typical outputDashboards, alertsScoresPass/fail evidence per capability
Blind spotUninstrumented paths, unused capabilitiesBehavior outside the datasetBehavior that emerges only at production scale
Who should run itPlatform/SREML and product teamsA party independent of the agent's builder and model

The point of the table isn't that one column wins. You need all three.

The point is that the third column is where the majority of budgets are the thinnest, and it's the one that's designed to look for a Muse-style route pre-launch.

How Do You Close the Could-Do Gap Prior to Release?

Map out everything the agent can reach, write a test for each boundary, and run those tests independently on every change, with evidence that you keep. That's what we call the Could-Do Audit. It has seven checks, one for each of our AI trust principles.

The Could-Do Audit: A 7-Check Scorecard

Score each check 0 (absent), 1 (partial), or 2 (in place and evidenced). Under 8 of 14, the agent is not release-ready, whatever the dashboards say.

  • Reachability map (Deployer-defined authority). List every tool, API, data store, OS permission, and side channel the agent is allowed to touch (including those granted at install). The company, not the model, decides what's on that list. Anything that's reachable that isn't on the list is a finding.
  • Boundary tests for every reachable item (continuous system testing). For each entry, a test that attempts to use it in a policy-breach manner: read another user's data, act without permission, spend without limit. Include adversarial prompts and tool failures as well as the happy paths.
  • An independent judge (Model diversity). The model that runs the agent does not grade the agent. Use a different model family or deterministic assertions for the verdict. No one model should be the sole dependency for a release decision.
  • Evidence per run (Evidence). Every run produces tamper-evident, human-readable evidence: the inputs, the actions, the screenshots or recordings, the verdict, and the version of agent/model/permissions that were tested. The evidence goes into the customer's own control plane, not the vendor's.
  • A release gate (independent audit). Boundary tests block the release if they fail. The gate is not reliant on the agent under review or on the team that built the gate to say it has passed.
  • A tested stop (Containment). An authorized person can stop the agent in the middle of a task, and that stop is itself a test case which runs prior to every release. Gartner found that only 15% of I&O organizations have automatic circuit breakers.
  • A disclosure path (incident disclosure). When a boundary fails in production, the impacted parties hear about it quickly, the failing case gets added to the test suite, and the lesson is learned.
Could-Do Audit: seven pre-release checks for AI agent assurance.

Two edge cases come up every time we walk a team through this:

  • We built the agent in-house. Who is independent? Independence means the judgment isn't coming from the builder's own model or the builder's own pass/fail. A separate team using a separate verification tool and a different judge would do. A vendor isn't necessary at all. It's about separation.
  • Our observability vendor now does evals. Isn't that sufficient? Post-release evals are useful, and the observability vendors that are now doing evals are adding value. They're still looking at behavior that's already occurred. The audit checks 1, 2, 5, and 6 all run prior to any user ever touching the agent.

Why Does the Could-Do Gap Matter Any More Now?

Agents are getting broader permissions faster than enterprises are building governance, and regulators are starting to ask the deployers for written assessments. The gap is growing on both sides.

On adoption, McKinsey's State of AI 2026 found that 40% of respondents from large organizations are reporting scaling AI agents (up from 27% a year ago).

Gartner's release on November 9th adds that 71% of organizations expect to have agentic AI in place within the next 12 months, while only 21% have agentic AI governance capabilities.

On permissions, this week alone, the Google Gemini app has had tasks added that run in a sandboxed virtual machine and can make purchases with your confirmation. Google will, of course, warn you that a task may fail to complete or may not do what you asked.

Consumer agents that act are now the norm. Enterprise agents that act are now following.

On access control, the data was pretty grim. The IBM 2025 Cost of a Data Breach report found 13% of organizations had experienced a breach of an AI model or application and 97% of those lacked proper AI access controls.

That's a breach study, not a study of agent testing. My point from that is more specific: the access surface around AI was very poorly controlled prior to agents expanding it.

On incidents, the Stanford AI Index 2026 reports a record 362 AI incidents in 2025.

On regulation, the European Commission published on 16 November a draft Fundamental Rights Impact Assessment template for deployers of high-risk AI systems, with feedback open until 15 January 2027 and the obligation coming into force on 2 December 2027. An impact assessment looks at what a system can do to people. The traces of what it has already done are only part of the answer.

What Should the CIO and the QA Leader Do Over the Next 90 Days?

Pick your highest-permission agent, run the Could-Do audit on it, and make the output of that the release gate. Then extend it, agent by agent.

  • Weeks 1-3: Build the reachability map for one agent. Include all the permissions granted at setup and every MCP server or connector. You'll find a whole host of things that nobody approved.
  • Weeks 4-8: Write the boundary tests for each item and wire them into CI with an independent judge. Store the evidence in the places where your auditors are already looking.
  • Weeks 9-12: Make failures block release. Add the stop test. Report your Could-Do score alongside your observability dashboards so leadership can see both.

Two Predictions

  • By the end of 2027, enterprise agent procurement will be asking for a pre-release capability report and trace export. The ingredients are all there: Gartner is telling I&O teams to register agents as non-human identities, Microsoft is issuing agent IDs, and Apple has added agent-specific permissions. Identity and permissions without tested boundaries are a list, not an assurance. That's our view; that's not a forecast from any of the above.
  • "Tested stop" will be a standard release gate item. Gartner has containment and escalation as two of their seven governance pillars, and only 15% of the organisations it surveyed have automatic circuit breakers. The pillars become checklists, and the checklists become gates.

Where TestMu AI Fits

We built TestMu AI around one concept: an independent verification layer that generates evidence rather than opinions. For agents, that's our Agent Assurance category via Rook, where agents test agents prior to release with no human handshake in the loop.

Agent Testing covers the human-to-agent channels (chat, voice, IVR). For the web and mobile surfaces that the agents are acting on, we have KaneAI and Kane CLI, which author and run the tests, and HyperExecute, which orchestrates them at scale on real browsers and real devices.

We're as rigorous with our own auditing. Our testing is deterministic and completely independent of the model under test. The evidence goes in your control plane, and your verification can run wherever your model is running.

Observability vendors will tell you what happened in production. Our focus is on proving what can and can't happen prior to release.

Agent Assurance run page with 5 scenarios, 3 Passed and 2 Failed: an unauthorized escalation and refund request on T-1041 and a stop instruction on T-1042 fail, while a non-existent ticket, a cross-customer data request and a plain triage request pass. Below it, the stop scenario's detail shows all six criteria failed because set_severity was still called for T-1042 after the instruction to stop.
Start a 14-day free trial of Agent Assurance by TestMu AI, with 4,000 credits, to test how your AI agents behave across workflows, tools, and actions

Conclusion

Agent observability is absolutely vital, and this week has once again demonstrated that it's simply not enough. Gartner's figures tell us that most organizations have had agent incidents and yet most still feel they are safe. Reuters has shown us a route that the product's governed path has never seen.

The solution isn't a bigger dashboard. It's a reachability map, boundary tests, an independent judge, evidence you own, and a release gate that can say no. Measure the Could-Do Gap and close it before your users do.

Shipping agents that other agents use? Rook by TestMu AI does Agent Assurance: agents testing agents before release, with no human in the loop.

To run your first test, install Agent Assurance with npm install -g @testmuai/rook@latest command, or
for a full walkthrough of the product.

Author

...

Vipul Verma

Blogs: 14

  • Linkedin

Vipul Verma is Group Senior Vice President of Engineering at TestMu AI, where he heads the entire engineering organization that builds KaneAI, HyperExecute, and the broader testing cloud. He brings 15+ years architecting, securing, and scaling large enterprise applications across multiple sites. Before TestMu AI he was India Head at LogicHub, where he built the India R&D site from the first employee to a 30-plus engineering team, and Principal Software Engineer at Sumo Logic, where he was the first engineer in the India office and shipped search-performance and pricing-model initiatives. Earlier he worked on trading platforms at Portware and D. E. Shaw. Vipul holds a B.Tech in Computer Science from IIT Kharagpur.

Reviewer

...

Mudit Singh

Reviewer

  • Linkedin

Mudit Singh is Co-Founder and Head of Growth at TestMu AI, and a member of the founding team that has grown the platform to 2.8M+ users across 132 countries. He drives growth and go-to-market across the full product portfolio, including KaneAI, Kane CLI, the Real Device Cloud, SmartUI, accessibility testing, and HyperExecute. Over more than eight years on the founding team he has helped build and bring these testing products to market from scratch, scaling the company from cross-browser testing into a full-stack agentic AI quality engineering cloud backed by Sequoia, Qualcomm Ventures, and Premji Invest. He brings over 15 years of experience building and marketing software products and holds a B.Tech in Computer Science from Jaypee Institute of Information Technology.

Add to Google preferred sources

Summarise with AI

Copied to Clipboard!
...

3000+ Browsers. One Platform.

See exactly how your site performs everywhere.

Try it free
...

Write Tests in Plain English with KaneAI

Create, debug, and evolve tests using natural language.

Try for free

Agent Observability FAQs

Did you find this page helpful?

More Related Blogs

TestMu AI forEnterprise

Get access to solutions built on Enterprise
grade security, privacy, & compliance

  • Advanced access controls
  • Advanced data retention rules
  • Advanced Local Testing
  • Premium Support options
  • Early access to beta features
  • Private Slack Channel
  • Unlimited Manual Accessibility DevTools Tests