The RAISE Act Tests the Model - Who Tests Your Agent?

What Does the RAISE Act Actually Require?

The RAISE Act requires frontier AI developers to publish transparency reports when they deploy a new frontier model or a substantially modified frontier model and to report any critical safety incidents to New York State within 72 hours. The largest developers are also required to publish a safety framework.

The Act imposes its obligations on the developers of the models themselves and imposes no testing obligations on companies that build agents on those models.

The details, from the chapter amendment signed on March 27, 2026 (NY Senate, A9449):

• Scope: A frontier model is one that has been trained on more than 10²⁶ operations of compute (counting compute for fine-tuning and other material modifications). A large frontier developer is a frontier developer with more than $500m in revenue (including affiliates) in the prior calendar year.
• Safety framework: Large frontier developers must publish a frontier AI framework covering catastrophic risk.
• Transparency reports: All frontier developers must submit them either prior to or at the time of deployment of a new model or a model that is substantially modified.
• Incident reporting: All frontier developers (not only the big ones) are required to report any critical safety incident to the state within 72 hours. Where there is an immediate risk of death or serious physical injury, there is a requirement to report to an appropriate authority within 24 hours.
• Enforcement: The Attorney General can pursue civil penalties of up to $1m for the first offense and up to $3m on subsequent offenses.
• Registration: Large frontier developers must register via the RAISE Portal by 1st January 2027.

Read that list as a deployer, and the thing that jumps out is that pretty much every obligation falls on the lab. One of the provisions allows the state to fine you if you develop, deploy or operate a large frontier model in New York without a current disclosure on file, but again, that's a filing obligation; it's not a testing obligation.

None of it relates to the stack of prompts, retrieval sources, tool connections and permissions that actually turn the model into your agent.

Who Independently Tests AI Models These Days?

A growing number of public bodies are either testing or capable of testing frontier models: New York's new research center, the EU AI Office under Article 92 of the EU AI Act and, if Australia's proposal becomes law, the independent assessors under its proposed regime. All of them are looking at the models, not at the deployments.

Here is the landscape as of this week:

• New York: The new center will independently test advanced AI systems and will begin to review the published safety frameworks of developers as part of the work of the new DIGIT office, which is the office through which the DFS will be leading the implementation of the RAISE Act.
• Australia: Under the proposal, frontier AI companies wishing to operate in Australia would be required to allow for independent evaluation of their models and to rapidly notify the government of any incidents. A new regulator would work in conjunction with the Australian AI Safety Institute for technical expertise. Legislation is still to be drafted.
• European Union: Article 92 of the AI Act enables the AI Office to carry out an evaluation of a general-purpose AI model in order to assess compliance or to investigate systemic risks (EUR-Lex, Regulation (EU) 2024/1689).

This is important. Independent evaluation of AI models by public bodies breaks the 'lab marking its own homework' habit.

It's a prime example of one of our core AI trust principles: independent audit. An audit at the national level must not be reliant on the model being audited.

But note where the line is drawn. The NY announcement refers to frontier AI systems and frontier-scale developers. It says absolutely nothing about testing the applications that enterprises build on top of them.

Why Does a State-Tested Model Not Necessarily Mean a Safe Agent?

The risk in an enterprise agent comes almost entirely from what the deployer has added to it (system prompts, retrieval data, tool access, permissions, workflow logic and the other agents it interacts with). A public tester will never see that configuration, so can't possibly be in a position to vouch for it.

Think about what changes between the model a state lab evaluates and the agent your customer talks to:

• Instructions. Your system prompt either constrains or expands behavior in ways that a base model test cannot predict.
• Data. Retrieval sources bring in stale, conflicting, or poisoned data.
• Tools. An agent that can issue refunds, change records, and send messages is quite different from one that can only chat.
• Authority. Who decides what the agent can access? Our principle is deployer-defined authority: it's the company that decides what the model can access and do. That's your call. The test of that is your test.
• Neighbors. In agent-to-agent workflows, your agent's inputs come from other agents that nobody's ever worked together with.

The EU has already split this into law. Article 26 of the AI Act requires deployers of high-risk AI systems to put in place appropriate technical and organizational measures, assign human oversight to appropriate personnel and monitor the system's operation.

The standalone high-risk rules will come into application from 2nd December 2027 and the product-embedded ones from 2nd August 2028 (European Commission).

The obligation is on the deployer. There is no public tester coming to run it for you.

The incident curve is going the wrong way. The Stanford AI Index 2026 reports 362 incidents in 2025 (up from 233 in 2024). The same report (citing McKinsey survey data) shows the proportion of organizations rating their incident response as 'excellent' has dropped from 28% in 2024 to 18% in 2025 (Stanford HAI, AI Index 2026).

Deployment is moving very quickly. McKinsey's State of AI 2025 found that 23% of respondents said their organizations were scaling an agentic AI system and a further 39% were experimenting with agents. Separately, 51% of respondents from organizations that were using AI reported at least one negative consequence (McKinsey 2025).

If trust can't be demonstrated, then businesses go down the route of blocking. On 12th November, United Airlines announced it is moving to block unauthorized third-party personal AI agents from its booking and ticketing channels for security reasons (exposed credentials, unintended purchases, etc.) (TechCrunch, 2026).

That's what an AI trust gap looks like in the real world: no way to prove that an agent is behaving in a trusted manner, and so the door is shut.

What Is the Three Line Test Model for AI Assurance?

The Three Line Test Model breaks AI assurance down into three distinct tests: the Lab Test (the developer tests their own model), the Public Test (a government or independent body tests the model), and the Deployment Test (the deployer independently tests their configured agent). Each line covers a risk that the others cannot.

We define it this way:

The Three Line Test Model is that an AI system is only trustworthy in production if all three of the following independent lines of testing have produced evidence of trustworthiness. A pass on one line of testing is not sufficient in itself.

The model isn't a hierarchy. Line 2 doesn't rank above Line 3. It's entirely possible for a model to pass every public test and still fail in your workflow, because the thing that failed was never the thing that was actually tested.

Stress-testing the model against edge cases:

• You build or fine-tune in-house. If your training plus fine-tuning compute exceeds 10²⁶ operations, you are a frontier developer under RAISE (a large one if you also had over $500m in prior year revenue) and both Lines 1 and 3 apply to you. The vast majority of in-house fine-tunes will not reach that threshold, but the Deployment Test still applies in full. Run it with a verifier that is not dependent on the model you are testing.
• You own the evidence and run the tests yourself (all good). Independence means the verifier isn't reliant on the model under review. It doesn't mean you outsource. Our model diversity principle is that no single model is the sole dependency for an important outcome and no model ever verifies its own work.
• Your vendor does have built-in evals. Useful as line 1 evidence. A vendor scoring its own model on its own platform is still a Lab Test, however good the dashboard.

How Do You Run the Deployment Test? A 7-Point Evidence Checklist

Run the Deployment Test as a release gate (i.e. before an agent ships) to prove with replayable evidence that it stays within its authority, handles failure and attack and can be stopped. Then keep testing continuously because the model, the data and the tools all change after launch.

Use this checklist. Each item corresponds to one of TestMu AI's AI trust principles and is written in such a way that a test can pass or fail it:

• Model diversity. The checker is not the model being checked. Pass: no release verdict is dependent solely on the agent's own model or vendor.
• Evidence. Every meaningful agent action leaves behind tamper-proof, human-readable evidence. Pass: an auditor can simply replay any test run and see the inputs, the actions and the outputs.
• Continuous system testing. The agent is tested on every change and on a schedule (including against tool failures, bad data and prompt attacks). Pass: a regression suite runs in CI and on a cadence.
• Deployer-defined authority. Permissions are defined by you and are tested. Pass: the tests attempt to get the agent to go beyond its scope, and it refuses every time.
• Independent audit. The audit tooling makes no use of the model under review. Pass: audit results reproduce without having to call that model.
• Containment. An authorized person can pause or stop the agent at any point. Pass: the stop control is exercised in tests, not simply documented.
• Incident readiness. You can let the affected parties know straight away. Pass: the evidence from item 2 is sufficient to reconstruct what happened so forensics don't delay the disclosure.

In practice, this means a release gate for agents, with evidence that ends up in your control plane and verification that runs where the model runs.

How Does TestMu AI Fit in With the Deployment Test?

TestMu AI provides an independent verification layer for Line 3. It tests what the agent does in your config and produces evidence rather than opinions.

We see three kinds of software interaction, and each needs its own Deployment Test:

• Human to software (web and mobile): KaneAI enables teams to write tests in natural language, and Kane CLI brings the entire quality engineering lifecycle to the terminal for SDETs and developers.
• Human to agent (chat, voice, IVR, WhatsApp, video): Agent testing checks agents against real conversation scenarios prior to release.
• Agent to agent: Agent Assurance (delivered by Rook) has agents testing agents with no human handshake prior to production.

Underneath, HyperExecute orchestrates those runs at scale. TestMu AI's test execution is deterministic, which is what makes its evidence usable in an AI audit.

We're held to the same model. The verifier has to be a different model, and the evidence has to be something that your auditors can read without having to trust us.

What Happens Next? Three Predictions

• Model-level testing will spread, and the deployer gap will spread with it. NY's rules come in on 1st Jan 2027, and Australia has proposed a mandatory regime. Vendors will be using public test results as part of their sales pitch. Buyers will be asking very quickly, "and who tested our configuration?"
• By the end of 2027, enterprise AI procurement will be asking for deployment-level evidence in addition to the model documentation. EU Annex III deployer obligations come into force on 2nd December 2027. For any team that already has the replayable Deployment Test evidence on file, they'll already have the answers.
• Unverified agents will keep getting blocked. United's move this week won't be the last. Businesses that let outside agents act on their systems will want proof of behavior, and that proof will come from Line 3 testing.

Conclusion: The RAISE Act Raises the Floor, Not the Ceiling

The RAISE Act and New York's new testing center are very real progress for AI safety testing. They provide a second layer of testing beyond the labs' own claims.

But the RAISE Act regulates the models. Your customers interact with the agents. The space in between (your prompts, your data, your tools and your permissions) is yours.

So is the evidence that it works. Run the Deployment Test before someone asks you for it.

Shipping agents that other agents use? Rook by TestMu AI does Agent Assurance: agents testing agents before release, with no human in the loop.

To run your first test, install Agent Assurance with npm install -g @testmuai/rook@latest command, or

book a demo

for a full walkthrough of the product.