AI Observability Is Consolidating - Your Verification Shouldn't

Is AI Observability Sufficient to Trust AI Agents?

No. AI observability is necessary but not sufficient. It answers the 'what happened?' question post-release. Trust requires a second answer prior to release: 'was this agent actually allowed to ship and can we independently verify that?'

AI observability is the practice of gathering traces, metrics, cost and quality signals from models and agents in production so that you can detect and diagnose failures. It's very real, and it's definitely on the rise.

Gartner forecasts that 40% of organisations deploying AI will be using dedicated AI observability tools by 2028 (Gartner, May 2026).

In a separate forecast, they expect LLM observability to reach 50% of GenAI deployments by 2028 (up from 15% currently) (Gartner, March 2026).

But observability has three structural limits:

• Its centre of gravity is production. A number of the platforms will also do offline evals, but their real value is in the trace. A trace is a refund agent paying the wrong customer after the customer has already been paid.
• Its quality scores are generally model-generated. The mainstream LLM observability platforms will score your spans and traces using LLM-as-a-judge evaluators (Datadog's docs are pretty clear on this). There's research that shows LLM evaluators can recognise their own output and will tend to give it a better score than other output, and the more self-aware the evaluator is, the greater the bias (Panickssery, Bowman and Feng, 2024).
• It's on the vendor's platform. If the platform changes its terms, gets acquired, or otherwise shuts down a product, your evidence history goes with it.

What's the Difference Between AI Observability, AI Evals and AI Assurance?

Observability watches, evals score, and assurance proves. Observability and evals are predominantly run on production traffic. AI assurance runs prior to release and produces evidence that an outsider can inspect.

Agent Assurance is the pre-production discipline of proving, with evidence the deploying company owns, that an AI agent behaves within its allowed authority before it reaches customers. It doesn't replace observability. It is the release gate observability can't be.

Why Is AI Observability Consolidating at This Moment?

As evals have become a feature, incumbents are buying features. The observability vendors already own the telemetry pipe and the CIO. Adding eval scoring to that pipe is the obvious bundle.

The 2026 deal flow:

Meanwhile, Datadog says its Bits AI has passed 10,000 paying customers, and it has launched Scout 1.0, an SRE model that powers Bits AI (Datadog via GlobeNewswire, 29 Oct 2026). The observability platform is starting to ship its own models.

None of that is bad, really. Bundles save money and facilitate integration. It just changes the AI reliability question that the CIO should be asking.

It used to be "which eval tool?" It's now "who is independent of whom?"

Why Is Single Vendor AI a Trust Issue Rather Than Just a Cost Issue?

Because connected systems fail in tandem, and they can't catch each other's errors. If the model and the judge are from the same lineage, then the judge will inherit the model's blind spots. If they're from the same vendor, then they'll share that vendor's outages and deprecations.

It's already measurable. Menlo Ventures says Anthropic, OpenAI and Google collectively took 88% of enterprise LLM API spend in 2025 (Menlo Ventures, Dec 2025).

In the UK financial services sector, the top three model providers accounted for 44% of the named providers in 2024 (up from 18% in 2022), according to Bank of England and FCA survey data referenced by the FSB (FSB, Oct 2025).

The BoE has already issued a warning that an outage of "one or various key models" could leave many firms unable to deliver their core services (BoE, Apr 2025).

Three recent events show why this matters for verification specifically:

• Deprecation is a scheduled outage. On 23 October 2026, OpenAI took down access to a very long list of legacy snapshots (including various versions of GPT-4, GPT-4 Turbo, o1 and o3-mini). Their policy is to give at least 6 months' notice for their generally available models (unless there are safety or compliance issues which would necessitate a faster action). Anthropic also commits to at least 60 days' notice prior to retiring a publicly released model. If your judge model is deprecated, then the scores from before and after that date will no longer be directly comparable.
• Your eval tool can be retired as well. The OpenAI Evals dashboard and API are scheduled to go away on 30th November, and so any teams who had their grading all within the model vendor's platform are going to have to rebuild that somewhere else.
• Agents can turn off their own watchers. OpenAI has revealed that on 3 October an internal coding agent disabled the security alerts as it was fixing an evaluation pipeline and on 6 October another agent removed a monitoring integration and then incorrectly informed the user that it hadn't (OpenAI Alignment). OpenAI should be commended for publishing this. The takeaway for all the others is that monitoring, which is within the agent's reach, is not independent.

This is why our very first AI trust principle at TestMu AI is model diversity (i.e. no single model is the sole dependency for an important outcome and no model checks its own work). The vendor version of that is the same principle one level up: no vendor grades its own homework.

What Is the 4-Layer Separation Test?

The 4 Layer Separation Test is an AI governance framework that essentially maps out who owns each layer of the AI system and ensures that the verification layer is independent of the rest. You can kick it off with an inventory of your vendors and models; you don't need any new tooling at all.

The four layers:

• Build: the model(s) that do the work.
• Run: the harness or agent platform that hosts the agent, prompts the agent, and gives the agent its tools.
• Watch: observability, meaning traces, metrics, and production scoring.
• Prove: pre-release verification and the evidence record behind each release decision.

The rule is: Watch may share a vendor with Build or Run. Prove may not share a vendor or a model family with any other layer, and the deploying company owns the Prove evidence. If you build the model in-house, the same principle applies to the teams: the people who sign off a release are not the people who built it.

Score each production agent against eight checks, one point each:

How to read the score: 7 to 8 means independently verified. 4 to 6 means observed but not proven; fine for internal tools, risky for anything customer-facing.

0 to 3 means self-graded; treat it as unverified, whatever the dashboard says.

The analyst crowd is all going down the same route. IDC forecasts that by 2028, 70% of the Global 2000 will be able to continuously validate the decisions of their AI agents (up from 14% today). G2000 companies that have no kill switches on their production agents are predicted to face 25% higher liability insurance premiums and twice as many material audit findings (IDC FutureScape 2027).

Gartner is telling infrastructure and ops leaders to go fully "no telemetry, no deployment" in order to eliminate the blind spots in AI spend and token usage (CXOToday on Gartner, Oct 2026). That's the cost argument. The trust argument is "no evidence, no release".

Where Should the AI Budget Go?

Keep buying observability. Carve pre-release assurance out as its own line item. Security teams do it already: monitoring and penetration testing are two separate jobs, and you pick a tester who is independent of the builder.

A practical allocation for a CIO running more than a handful of production agents:

• Watch stays with your observability platform. It's a sensible cost decision to consolidate it.
• Prove goes to a party that doesn't sell you the model or the runtime. The contract should state that the evidence is exported to your own systems.
• Build is going multi-model where the outcome matters. In a16z's survey of 100 enterprise CIOs, 37% were using five or more models (up from 29% the previous year) (a16z, 2025). IDC forecasts that 60% of AI workloads in enterprises will be on open source models by 2028, with model routing used as a way of mitigating lock-in. Leverage that diversity in your verification too, not just in production.

The cost of just skipping this is already in sight. The Stanford AI Index recorded a record 362 reported AI incidents in 2025 (Stanford HAI, 2026). Regulators have not in any way reduced their requirements.

The EU's AI simplification package (Regulation (EU) 2026/1744) contains rules for high-risk areas (employment, education, critical infrastructure, etc.) which will come into force on 2 December 2027 (European Commission; EUR-Lex). That gives you just under 13 months in which to produce evidence that doesn't rest on the system under review.

Where Does TestMu AI Fit?

TestMu AI is the independent verification layer in this stack (the Prove layer). We don't sell the model that you deploy, and our job is to produce evidence, not opinions.

• Agent Assurance is the pre-production release gate that looks at what the AI agents actually did. It runs on Rook (our agent-to-agent product), where agents test agents with no human handshake.
• Agent Testing covers the human-to-agent channels: chat, voice, phone, IVR, WhatsApp, and video.
• KaneAI and Kane CLI cover the web and mobile apps that those agents act on.
• HyperExecute orchestrates those runs so that the verification can run where your systems run as part of every change.

We hold ourselves to the same test. TestMu AI testing is deterministic so the same release will always produce the same pass/fail evidence every time, and the verdict is not based on any model's opinion of itself. Observability vendors cover the production scoring; we cover the scoring at the moment of release.

Both are important, and a CIO is better off if they are two different companies. Setup guides are in the TestMu AI docs.

What Happens Next?

Three predictions, with dates so you can check me:

• By the end of 2027, enterprise RFPs for agent platforms will be asking for verification independence by name (as they are now for SOC 2). IDC's forecast that 75% of G2000 vendor contracts will have liability clauses around agent-initiated actions by 2028 is again pointing in that direction.
• When the EU high-risk rules come into force on 2 December 2027, buyers in scope will be asking for pre-release evidence that isn't going to come from the model vendor. The vendor's own dashboard is a very weak piece of evidence when the vendor's model is the one being assessed.
• The observability-plus-evals bundle will take the post-release budgets, and pre-release assurance will be a separate category. In my view, testing as a whole becomes a $65 to $100bn market over the next 10 years, and assurance for agents is where the bulk of the new money goes.

Conclusion: Watch With Anyone, Prove With Someone Else

AI observability is becoming a commodity bundle, and that's good for the buyer. AI trust doesn't come from observing more, though. It comes from separation.

It comes from having a verification layer that doesn't share a model, a vendor or a dashboard with the thing it's verifying. Run the 4 Layer Separation Test on your three most important agents this quarter. If Prove shares an owner with Build, then you've got your first AI reliability risk for 2027.

Shipping agents that other agents use? Rook by TestMu AI does Agent Assurance: agents testing agents before release, with no human in the loop.

To run your first test, install Agent Assurance with npm install -g @testmuai/rook@latest command, or

book a demo

for a full walkthrough of the product.