How to Audit AI Agents: An AI Audit Guide for Agent Workflows

What Is an AI Audit?

An AI audit is a formalised check that the AI is behaving as it should and that the people responsible for it can demonstrate that. (The term is also used in the context of AI tools which assist accountants in the audit of financial statements. This guide is about auditing the AI itself.)

For a classic model, the audit was about training data, accuracy, and bias. For an agent, it's much more.

The agent takes action: it updates records, it sends messages, it moves money, it closes tickets. The audit has to be about what the agent actually did, not just what it said.

AI Audit Frameworks, and the Gap for Agents

The regulatory frameworks (NIST AI RMF, ISO/IEC 42001, the EU AI Act) all set the expectations in terms of what you need to do: assess risk, document the system, keep humans in control, monitor it over time. They tell you what to govern.

They don't tell you how you prove that a particular agent/workflow actually did the right thing on a particular run. That proof has to come from testing the workflow itself.

Why Agents Break the Old Audit Model

Three things make agent workflows more difficult to audit than the software that came before.

They act across systems. One workflow can read from a CRM, write to an ERP, and open a ticket. The evidence of what happened is dispersed across all three.

The checks are getting increasingly baked in. Model platforms now come with their own reviewers and guardrails. That's useful in its own right, but when the vendor that built the agent has also built the check and holds the logs, then the audit trail is not independent.

Behaviour varies run to run. The same agent given the same task can take very different paths. A one-off review of a single run tells you very little about the next one.

Five Principles for Auditing AI Agents

1. The model doesn't validate its own work. The verifier must not have access to the agent's model, vendor, or context. A second opinion from the same source is not an audit.

2. Audit the outcome, not the actions. The logs will tell you what the agent tried to do. The audit has to tell you what actually changed: what records, what values, what downstream effects.

3. Verdicts must be repeatable. If you run the same check on the same workflow, you should get the same answer. That's deterministic testing against a written spec rather than a score that will vary from run to run.

4. Evidence is the deployer's. Any meaningful action must leave tamper-evident, human-readable evidence in the company's own control plane rather than just in the vendor's logs, which can be altered or simply go away.

5. The company sets the rules. What the agent is allowed to access and what constitutes a correct answer are decisions of the business that is deploying it, specified in writing prior to release and not something that the model infers at run time.

An AI Agent Audit Checklist

What a single piece of audit evidence looks like. Every verified run should leave behind a record that an auditor can just read without having to ask an engineer.

Where Agent Assurance Fits

Agent Assurance from TestMu AI is designed to generate that evidence prior to release. Rook's test agents run each workflow against deterministic specs, check the end state of every system they've interacted with and record the verdict in your own control plane. The verifier is completely independent of the agent's model and vendor so it's not grading its own work.

In audit terms it ticks the middle four rows of the checklist before the workflow has even reached a customer. Guardrails and production monitoring are still very much the thing in terms of what happens during the run and what happens post release. Agent Assurance is the one that covers the decision in the middle i.e. should this ship?

Conclusion

The rule that makes a financial audit worth trusting applies to agents too: the thing being audited can't audit itself.

Frameworks like NIST AI RMF, ISO/IEC 42001 and the EU AI Act tell you what to govern, and vendor guardrails keep individual runs in bounds. Neither gives you proof that a specific workflow did the right thing.

Agent Assurance produces that proof before release: an independent verifier checks the end state of every system the agent touched against a written spec, and the verdict lands in your own control plane, ready for an auditor to read.