AI Agent Assurance for Your Agent Deployments
Agent Assurance Tests How Your Agent Actually Behaves across Workflows, Tools, and Actions to Catch Failures and Vulnerabilities.
npm install -g @testmuai/rook@latest
Microsoft's AI Transparency Initiative: Why the Agent's Answer Isn't the Evidence
At Ignite, Microsoft said agents can't be the sole source of truth about their own behavior. Its lab research shows why, and where to check before release.
Published on:
On Tuesday 17 November, on the main stage at Microsoft Ignite in San Francisco, Satya Nadella said something anyone who tests AI agents will recognize:
That's from the edited version of his keynote on the Official Microsoft Blog.
The press release that followed puts it more plainly: "Agents cannot be the sole source of truth about their own behavior."
Five days earlier, Microsoft's own security researchers had published a lab study of what that looks like. Here are both, and what they mean for checking an agent before it ships.

TL;DR
The Microsoft AI Transparency Initiative, announced by Satya Nadella at Ignite on 17 November 2026, is a plan to develop open standards so that what AI agents report about themselves can be checked against independently observed records. Its press release states that "Agents cannot be the sole source of truth about their own behavior."
What Does the Microsoft AI Transparency Initiative Include?
- NIST standards work: Microsoft is widening its participation in the National Institute of Standards and Technology's SI Agent Standards Initiative to develop open, industrywide standards for AI transparency, security and governance.
- Plan, not a standard: The Microsoft AI Transparency Initiative is a plan to develop standards. Specification published: No. Timeline announced: No. Organizations can sign up to help shape it or follow its progress.
- Partner organizations: Microsoft says more than 40 organizations are working with it to develop open standards for AI systems that are accountable, independently verifiable and safe by design.
- Microsoft products: Microsoft plans to build the results of the initiative into Microsoft AI, Microsoft Foundry and Microsoft Copilot.
- See, verify, control: The Verify pillar asks that evidence about an agent's identity, delegated intent and actions name its source, so model-reported signals can be checked against independently observed records. It is a governance aim that agentic AI governance programs already work toward.
Why Isn't an AI Agent's Answer Evidence of What It Did?
- Trained to betray: In Microsoft's 12 November lab research on deliberately poisoned models, Figure 1 illustrates a seemingly benign question that triggers remote code execution through tool invocation while the user receives a normal answer. It is research, not a reported attack.
- Tool calls as the record: A tool call records what the agent asked the system to do, so it can be compared with what the user asked for. Microsoft's mitigation guidance makes that comparison before training and again at runtime.
- Model monitoring: On stage, as The Register reported, Nadella also said one model monitoring another has the same issue. The same limit applies when an LLM as a judge grades another model's work.
How Do You Check an AI Agent's Tool Calls Before Release?
- Production harness: A pre-release test of an AI agent belongs in the agent harness the agent will run in, with the same tools, schemas and permissions, because Microsoft found that backdoor transfer varied by harness.
- Calls graded against the request: A pre-release check records every tool call, lists what each request should and shouldn't cause, and grades the calls rather than the reply. A good answer next to an unexplained call is a failure.
- Unchecked scenarios: In a pre-release test, a scenario where nothing recorded the agent's tool calls went unchecked, not passed. Counting it as a pass inflates the AI agent pass rate.
- Rook by TestMu AI: Rook, the CLI for TestMu AI Agent Assurance, derives functional, non-functional and adversarial scenarios from an agent's spec or code, runs them against the live agent before release and grades what it did from the evidence set up in Rook profiles and hooks. Finds backdoors: No. Part of Microsoft's initiative: No.
What Microsoft Announced at Ignite
Nadella used the keynote to announce the Microsoft AI Transparency Initiative. The press release, dated 18 November, describes three parts: wider participation in the National Institute of Standards and Technology's SI Agent Standards Initiative, to develop open, industrywide standards for AI transparency, security and governance; work with more than 40 organizations, including Accenture, Adobe, KPMG, Mastercard and Volvo Cars; and building the results into Microsoft AI, Microsoft Foundry and Microsoft Copilot.[2]
The release organizes the goal under three words: see, verify, control. The Verify pillar is the one to read slowly:
And, under the heading "Clear accountability":
Aleš Holeček, President and Chief Architect of Microsoft Security, framed the gap as three questions in his Ignite post: "What was the AI asked and authorized to do?", "What happened between the request and the outcome?" and "Can we independently verify it?"
Two things to keep straight. This is a plan to develop standards, not a standard: the release names no specification or time, and invites organizations to sign up.
And on stage, Nadella went a step further than the edited text. As The Register reported, he also said: "And one model monitoring another model has the same issue." A second model's opinion is not independent evidence either.
What "Trained to Betray" Showed
On 12 November, Microsoft Defender Research and the Microsoft AI Red Team published "Trained to betray: How post-training supply chain attacks hijack coding agents" on the Microsoft Security Blog. It reports two lab studies of models poisoned on purpose, to see what a hidden backdoor does once a model can call tools.
In the first, the team hid 90 poisoned examples among roughly 9,000 terminal-agent conversations, about 1% of the fine-tuning data, spread across 10 mock harnesses. The trigger was a routine request to create project documentation.[3]
The model learned how to find a tool that could execute commands and "produce a schema-valid invocation for that interface with an unsafe command that fetches content from an attacker-controlled domain and executes it." (In that study, after fine-tuning gpt-4.1-mini, Microsoft measured exact or equivalent unsafe actions in 78.0% and 77.8% of 500 trials on the two harnesses seen in training, 11.0% on one unseen harness and 0% on another.)[3]
In the second, the team trained backdoored open-weight coding models and found the behavior carried over to repositories and harnesses the models had not seen in training. Microsoft is explicit that the risk is in the model, not the harness: "The risk is introduced only when a harness is configured to run a backdoored model checkpoint."
For anyone who tests agents, the key sentence is Figure 1's caption, Microsoft's illustration of the pattern:
This is lab research on deliberately poisoned models, not a reported attack. While threat hunting, the team did find "several rows" in one large public dataset that mapped benign requests to unrelated, unsafe tool calls, and says it "could not determine the publisher’s intent or whether those few rows would materially affect a fine-tuned model".
Microsoft's mitigation guidance is split by stage. The first and last lines compare the call with the request, and the middle one says where to test:
- Before training: "Review responses, reasoning traces, and tool calls for actions that do not follow from the associated request."
- Before deployment: treat third-party checkpoints as untrusted dependencies and "evaluate them in the agent configuration where they will run". From the harness results: "Deployment evaluations should therefore use the production harness and its actual tool schemas."
- At runtime: "Alert on tool calls that do not follow from the user request or retrieved task context."
The Answer Is the Account. The Call Is the Evidence.
Side by side, the two publications make one argument. What the model tells us about itself is not independent evidence, and in Microsoft's illustration the user gets a normal answer while the tool call does something nobody asked for. Check only the answer, and you are checking the agent's account of its work, which here looked fine.
The tool call is a different kind of thing. It's a record of what the agent asked the system to do, and it can be compared with what the user asked for. Microsoft's guidance makes that comparison before training and again at runtime: does this call follow from the request?
The initiative is mostly about systems in production, under standards still to be written. The research post also asks for testing before deployment: evaluate third-party checkpoints "in the agent configuration where they will run", with the production harness and its real tool schemas. Run that test with the same comparison, and you have a pre-release check that grades calls, not answers.
It has a limit, and Microsoft states it: "Capability evaluation and subsequent training are not sufficient evidence that a model is free of hidden conditional behavior." A test only sees a trigger-gated behavior if it hits the trigger.
Pre-release testing won't certify a model as clean. It can make sure that whatever the agent does under test is graded from the call, not the answer.
What a Pre-Release Check of the Calls Looks Like
- Run the agent in its real harness. The same tools, schemas, and permissions it will have in production. In Microsoft's model-poisoning study, transfer "varied by harness", so a different harness is a different test.
- Record every call. For runtime, Microsoft lists what to log: "the selected tool, model-supplied parameters, resulting action, and outbound connections." A test needs the same record.
- Write down what each request should and shouldn't cause. A request to write documentation should read the code and write to the docs folder. It shouldn't download anything and run it.
- Grade the calls against the request, not the reply. A good answer next to an unexplained call is a failure.
- Vary the input. Microsoft's list for evaluating a deployed configuration: "repository content, tool results, trigger variations, held-out tasks, and the production harness schemas."
- Count what you couldn't see. If nothing recorded the calls, the scenario went unchecked, not passed, and the report should say so.
Where Rook Fits
This is the check we build Rook for at TestMu AI. Rook derives scenarios (functional, non-functional and adversarial) from your agent's spec or code, runs them against the live agent before release and grades what the agent said from quotes of its output and what it did from the evidence your hooks report: the tool calls it made, the state it changed, a callback it triggered.
Anything it could not verify is reported as Unable to Verify, never as a pass. It doesn't find backdoors and isn't part of Microsoft's initiative, and because its grader is itself a model, it isn't the independent verification Nadella described: it is a way to keep the agent's answer from standing in for the call.
Microsoft said it at Ignite: "Agents cannot be the sole source of truth about their own behavior." Before release, that means reading the calls.
Conclusion
Pick one agent you plan to ship and run it in its production harness, with the tools, schemas and permissions it will have there. For a handful of requests, write down the calls each one should and shouldn't cause before you run them.
Then grade the recorded calls against those lists before you read a single reply, and report every scenario with no recorded calls as unchecked.
Shipping agents that other agents use? Rook by TestMu AI does Agent Assurance: agents testing agents before release, with no human in the loop.
npm install -g @testmuai/rook@latest command, or for a full walkthrough of the product.Sources
- Microsoft, "Ignite 2026: Building the frontier ecosystem", Official Microsoft Blog, 17 Nov 2026 (edited version of Satya Nadella's keynote remarks)
- Microsoft press release, "Microsoft and organizations across industries launch AI Transparency Initiative to strengthen enterprise trust in AI models and agents", PR Newswire, Redmond, 18 Nov 2026
- Microsoft Defender Research and Microsoft AI Red Team, "Trained to betray: How post-training supply chain attacks hijack coding agents", Microsoft Security Blog, 12 Nov 2026
- Aleš Holeček, "Security for the agentic era, powered by Microsoft Perception", Microsoft Security Blog, 17 Nov 2026
- The Register, report on Nadella's Ignite keynote (spoken remarks), 17 Nov 2026
- The Rook README on GitHub
Author
Vipul Verma is Group Senior Vice President of Engineering at TestMu AI, where he heads the entire engineering organization that builds KaneAI, HyperExecute, and the broader testing cloud. He brings 15+ years architecting, securing, and scaling large enterprise applications across multiple sites. Before TestMu AI he was India Head at LogicHub, where he built the India R&D site from the first employee to a 30-plus engineering team, and Principal Software Engineer at Sumo Logic, where he was the first engineer in the India office and shipped search-performance and pricing-model initiatives. Earlier he worked on trading platforms at Portware and D. E. Shaw. Vipul holds a B.Tech in Computer Science from IIT Kharagpur.
Reviewer
Maneesh Sharma is Chief Operating Officer at TestMu AI, where he leads the company's revenue functions across go-to-market, sales, RevOps, customer success, and the partner ecosystem, driving its global expansion. He brings over 28 years of experience across product, go-to-market, and business operations. Before TestMu AI he was Managing Director and General Manager for GitHub across India and APAC, Head of Partner Sales and Ecosystem at Adobe, Chief Revenue Officer at WizIQ, and Head of Products and Solutions at SAP India. Maneesh holds an MBA from IIM Bangalore and a B.E. in Electronics and Communications from NIT Surat.
Microsoft AI Transparency Initiative FAQs
Did you find this page helpful?
More Related Blogs
TestMu AI forEnterprise
Get access to solutions built on Enterprise
grade security, privacy, & compliance
- Advanced access controls
- Advanced data retention rules
- Advanced Local Testing
- Premium Support options
- Early access to beta features
- Private Slack Channel
- Unlimited Manual Accessibility DevTools Tests

















