AI Agent Incidents Don't Look Like Outages: theCUBE Research Findings

Why an Agent Incident Is Hard to Spot

A conventional failure announces itself: something goes down. The study's point is that an agent incident may not.

"The infrastructure can remain available, the data intact, and the agent properly authenticated." An agent "can use legitimate permissions to make a bad decision, execute it across several systems, and leave the business in a state no backup can fully unwind."^[1]

Security teams look for compromised identities, malicious activity and policy violations. An agent can be properly authenticated, use approved tools, stay within its permissions and still create unintended risk.^[1]

Detecting anomalous agent behaviour was among the least mature capabilities in the study: 12 of the 25 organisations had not addressed it or were not planning to. Meanwhile, as the report puts it, "the underlying systems may continue operating normally while the agent changes a record, initiates a transaction, sends a communication, or triggers activity elsewhere."^[1]

Human Approval Is Carrying the Load

For now, organisations are compensating with people. Among the 19 with agents in production, none allowed agents to execute high-risk actions without human involvement.^[1]

Eleven use bounded autonomy: agents act alone on lower-risk work and need approval for higher-impact activity. Eight require human approval for every action.^[1]

The study expects that to get harder to sustain, because "the speed and volume of agent activity will increase faster than the organization's capacity to review individual decisions." It also found some optimism: the bounded-autonomy organisations were more likely to report investment in logging, monitoring and recovery infrastructure, and more likely to have exercised agent incident scenarios.^[1]

Stopping an Agent Isn't Recovering From It

Circuit breakers and automated shutdown were among respondents' priorities for the next 12 to 24 months. The report is clear about their limit: they "can limit further activity, but they cannot undo actions an agent has already taken."^[1]

One respondent questioned whether hitting a kill switch could cost more in recovery effort and broken processes than it saves. Model fallback keeps users working when a model is unavailable, but "continuity does not resolve the effects of an agent's prior actions."^[1]

The Evidence Problem

Recovery starts with knowing who, or what, took an action, and that is where the study found some of its weakest results.

Agent identity is still immature. Several respondents said agents should run under separate service accounts, and one described what happens when an agent inherits a user's permissions: logs may show that an account made a change without making clear whether the user or the agent did it.^[1]

Reconstruction is harder still. Nine of 25 respondents were confident or partially confident they could reconstruct agent actions across systems; 11 lacked confidence or acknowledged a gap.^[1]

One described "so many disparate logs and audit trails" that getting to the core of what happened becomes difficult. Another, who was confident, qualified it: their logging was sufficient largely because their current agents could do very little damage.^[1]

The Largest Gap: Can the Business State Be Trusted?

Only six of 25 were confident or partially confident they could establish a last known trusted state at the business-process level; 16 acknowledged a gap or uncertainty. On remediation, 11 reported some capability to remediate agent-driven changes across multiple systems, and 10 acknowledged gaps or significant difficulty.^[1]

Some effects can be rolled back. Others can't: "A customer communication that has been read cannot be unsent. Information disclosed externally cannot be undisclosed."^[1]

So the report argues that the reversibility of an action should factor into how much authority an agent receives.

What the Study Recommends

Most organisations still have time. AI is not yet a critical operational dependency for most of them, and manual fallback is still available. That, the report says, is a window "to test agent failures while established processes and human expertise can still provide fallback."^[1]

Its suggested exercise: give an agent legitimate credentials and authorised access, have it make a bad decision across several functioning systems, and make the teams identify it, contain it, reconstruct what it did, trace downstream effects, reverse or compensate, and establish a trusted state. And test the recovery infrastructure itself: logs that stay available and trustworthy, identity records that preserve attribution, visibility across every system the agent can touch.^[1]

It's a careful report, and the exercise is a good one. Run it.

Where a Test Before Release Fits

Every step in that recovery chain starts from the same question: what did the agent actually do?

Without good reconstruction, the answer comes from whatever the agent left behind: its summary, its "done", its explanation of each step. That's the point we keep coming back to at TestMu AI: an agent's account of what it did is the weakest evidence available about what it did. It's written by the one party in the incident whose judgement is in question.

The recovery exercise tests what happens after the bad decision. There is an earlier test: does this agent make the bad change at all, under conditions you control? For the actions the report says can't be undone, like a sent message or an external disclosure, the only safe place to find that out is before release, in staging.

That's the step Rook, our agent testing tool, covers:

• You point it at staging, never production. Its tests make real changes, and it can't roll them back.
• It drives the agent the way a user would, then checks what actually changed, through hooks you configure.
• It checks the tools the agent called against what each scenario says it should and shouldn't call, and what the agent told the user against what it did.
• Every check comes back Pass, Fail or Unable to Verify. Unable to Verify never counts as a pass.

That last verdict matters for the reconstruction gap too. Writing a test forces the question an incident team will face later: where would we see what this agent changed?

If nothing can show it, the test says Unable to Verify. Better to learn that in staging than in the middle of an incident.

And to be plain about the limits: Rook doesn't test your recovery plan, it doesn't contain or roll back anything, and it doesn't block anything at runtime. It sits before the controls in theCUBE's report, not in place of them. It finds the bad change while the agent can still only make it in staging.

Sources

• [1] theCUBE Research, Krista Case, "AI Resilience: Organizations Are Underprepared and Largely Untested", 26 Oct 2026 (study with Qualitate of 25 enterprise technology and security leaders, for theCUBE's AI Cybersecurity and Resilience Summit, 27-28 Oct)