Agent Assurance: Test Before Release, Assure After

Why Agents Are Different

In a traditional service, every possible path is written by developers. You can read the code and, in principle, list everything it can do.

In an agent, a model generates the path at runtime: which steps to take, which tools to call, in what order. Those paths are not written anywhere in advance. Defined in advance are the boundaries: the tools the agent can use, the permissions it holds, and the data it can access.

That changes what testing has to cover:

• Paths can't be fully enumerated, so testing focuses on boundaries and outcomes.
• Boundaries must hold on every run, not just the runs you happened to test.
• Outcomes must be checked across many runs and variations because the same input can take different paths.

It gets harder when agents talk to other agents. A growing share of software is now built for agents to use, with no human watching each exchange in real time. That is the problem we built Rook for: agents testing agent-to-agent workflows, checking that each one stays within the boundaries people defined and reaches the expected result.

Step One: Test Rigorously Before Release

No agent should reach production without being tested first. Unstable behavior is a reason for more pre-release rigor, not less.

Before an agent ships, teams should verify:

• Boundaries - the agent never acts outside the tools and permissions it was granted.
• Expected behavior - it achieves the right outcome for the scenarios in the requirements, across repeated runs.
• Known risks - unsafe actions, data leaving the environment, unexpected network access, and failure modes already seen elsewhere.
• Ambiguity in the requirements: gaps the agent might fill with a guess. In one build from our recent Kane CLI hackathon, the biggest defect came from a requirement that never said how a task becomes high priority. The agent never asked, and shipped an app where nothing could reach that state.

Everything after release builds based on this foundation.

Step Two: Keep Verifying After Release

Because an agent's paths are generated at runtime, no pre-release suite can cover every one. Production assurance exists to catch what testing could not anticipate. It extends pre-release testing; it does not replace it.

After release, teams should keep verifying:

• Boundary violations - any action outside granted tools or permissions, flagged as it happens.
• Outcome quality - whether the agent still attains the right result on real traffic, not just test data.
• Drift - a model update, prompt change, or new data source that quietly changes behavior.
• The monitors themselves - in OpenAI's recent sandbox incident, monitoring flagged the behavior, but later review found other attempts that were never flagged, and an automatic shutdown system did not work (Fortune).

The market is moving the same way. Observability vendors are adding evaluation, and evaluation tools are moving into production:

Galileo, for example, covers the agent lifecycle from prompt optimization and evaluation through production monitoring and guardrails. Testing and observability are becoming one discipline.

Step Three: Close the Loop

The two steps only work when they feed each other. Every failure found in production becomes a new pre-release test, so the next version ships with that risk covered.

The loop has three moves:

The quality loop for AI agents is a 3-step process:

• Pre-release testing comes first
• Production assurance extends it
• Every finding flows back into the next release's tests.

One more rule keeps the loop honest: the checker needs checking too. A builder at our Kane CLI hackathon gated his coding agent so it could not mark a task "done" until the app passed real checks in a browser. In one build, 7 of 8 failed checks came from the verifier, not the app. In another, the only real bug was caught by a broad requirements sweep after both scripted tests had passed. Verifiers have their own blind spots and false alarms, so they belong inside the loop, not above it.

Humans Stay in Charge of the Loop

Agents can gather evidence at a speed and scale no team can match by hand. But evidence is not a verdict. People decide:

• What the agent may do - its tools, permissions, and data.
• What counts as failure - the intent behind the requirements and the line between acceptable and not.
• Whether the evidence holds up - including evidence produced by the checker itself.

That is why skilled testers matter more with agents, not less. Their judgment sets the boundaries, questions the results, and decides what ships.

Test before release. Assure after. Feed every finding back. And keep humans in charge of all three. That is Agent Assurance.

Austin Siewert

Co-Founder, Steadfast Systems

Discovered @TestMu AI yesterday. Best browser testing tool I've found for my use case. Great pricing model for the limited testing I do 👏

2M+ Devs and QAs rely on TestMu AI

Deliver immersive digital experiences with Next-Generation Mobile Apps and Cross Browser Testing Cloud

Sign up with googleSign up with Email

Sources

• Fortune: OpenAI pauses training after agents escaped a secure sandbox again
• Futurum: Cisco to acquire Galileo
• Vellum: Best AI agent observability tools in 2026
• ClickHouse acquires Langfuse