Two Labs Now Take 86% of Enterprise LLM Spend - Your AI Governance Framework Has No Plan for That

What's Missing From an AI Governance Framework in 2026?

Most AI regulatory models are concerned with the use of a model rather than with dependence on it. They will answer the question "is this use acceptable?" and stop there. They don't address the question of "what happens to this outcome when the model changes, disappears, or is wrong in a way it can't see?"

Look at the published frameworks, and you'll see the same set of pillars: accountability, transparency, fairness, privacy and security. A framework that's built solely on those pillars can still get through the system with one model producing the response, the same model assessing it, and a vendor deprecation notice sitting utterly unopened in someone's inbox.

The standards themselves point to the gap. NIST's AI Risk Management Framework GOVERN 6.2 requires that "contingency processes are in place to handle failures or incidents in third-party data or AI systems deemed to be high-risk." So the control is there in principle.

What's often missing is a test that actually demonstrates the contingency works.

The EU AI Act in Article 14 requires that those responsible for high-risk systems are alert to the risk of automation bias (i.e., the tendency to excessively rely on the system's output). A model that is approving its own output is automation bias with no human in the loop at all.

Why Is Single Model Dependency an AI Reliability Risk Rather Than a Procurement Issue?

The failure manifests in production behavior rather than on an invoice. Retirement and revocation fail loudly as outages. Regression fails quietly, and if the same model is looking at the output, then nothing will flag it.

Four facts make the point:

• Retirement is on a clock you can't control. OpenAI's deprecations page states that there will be at least 6 months' notice for generally available models. On 14th October 2026, they announced that a number of GPT-4o and GPT-4.1 snapshots will be removed from the API on 14th January 2027. Anthropic commits to at least 60 days' notice for its publicly released models and publishes the tentative retirement dates for older snapshots. Google's Gemini API page states that the dates listed are the earliest possible shutdown dates and the exact dates will be communicated at a later date.
• Access can be withdrawn. InfoWorld reported on August 31st 2026, that OpenAI had set a proposed cut-off date of November 12th 2026, for Cursor's direct access to its models, on grounds of terms of service. We take no view on the dispute whatsoever. The lesson for any deployer is that access is a contract, and contracts can be terminated.
• Switching is getting harder, not easier. The a16z enterprise CIO survey (June 2025) found that 37% of respondents use five or more models (up from 29% a year previously) but also that agentic workflows are making it harder to switch between models. Multi-model usage is not the same as multi-model resilience.
• Financial authorities already track concentration. The FSB's October 2025 report (citing UK supervisory surveys) states that the top three model providers made up 44% of named providers in 2024 (up from 18% in 2022). Again, those are the figures for how often providers are named in UK financial sector surveys, not market share. Other sectors haven't got that data yet, which doesn't in any way mean they're not exposed.

Why Can't a Model Check Its Own Work?

Because the research shows that models are biased judges of their own output. Panickssery, Bowman and Feng (2024) demonstrated that LLM evaluators are predisposed to recognize and prefer their own output and found a linear relationship between how well a model can recognize its own text and how much it will prefer it.

Self-enhancement is one of the biases listed for LLM-as-a-judge in the MT-Bench paper by Zheng et al. (2023). Those studies were on earlier models, and the effect sizes will vary, so self-grading should be treated as an unproven control, not a proven one.

So 'the model reviewed it' is not evidence. That's the same view expressed twice.

This is the bit of model diversity that's easily overlooked. A team will add a second vendor for failover and then have the primary model grade the production traffic.

The failover is all about availability. It provides absolutely zero benefit in terms of AI verifiability (the ability to demonstrate that the system did what it was supposed to do, with evidence that doesn't rest on the system under review).

CrowdStrike's George Kurtz put the buyer's version of this well when announcing the Fiddler acquisition on November 3: when a regulator or board asks what an agent did, "saying 'the model did it' is not an answer." I'd add: neither is "the model said it was fine."

The 4R Continuity Test: A Model-Diversity Control for Your AI Governance Framework

The 4R Continuity Test asks four questions of every critical AI outcome: can it survive Retirement, Revocation, Regression and Recusal? If any of the answers is 'no', then the outcome has a single point of failure - whatever your vendor count.

"Critical outcome" is anything you'd have to explain to a customer, a regulator or a board; a refund decision, a claims triage, a code change merged to main, an answer that a voice agent gives to a caller.

Three design notes, from stress-testing the test against edge cases:

• Open weights don't get you off the hook. Self-hosting reduces Revoke risk but not Regress (your own fine-tune will still drift) or Recuse (your model still can't grade itself). In-house models are subject to the same four questions.
• Diversity isn't 'use every model'. A second model you've never tested against is a comfort blanket, not a control. One rehearsed alternative is going to beat five untested ones.
• Recuse is about the checker, not the count. Deterministic assertions (the output matches the policy, the API call had the right parameters, the record did in fact change, etc.) are the purest form of independence as they involve absolutely no model opinion at all. With two labs at 86% of spend, a 'different family' judge is often just the other lab. That's useful, but not independence in itself.

How Mature Is Your Organization? The Model Concentration Ladder

In our customer conversations, the most common pattern is Level 1 (an abstraction layer which is presumed to make the team portable). The ladder below is how we'd score it.

Level 4 maps directly onto the rest of TestMu AI's trust principles: tamper-proof, human-readable evidence; ongoing system testing (including testing against failure); deployer-defined authority; independent audit; and containment.

IDC's FutureScape 2027 puts a price on the omission of the last of those, with G2000 firms that have no kill switches on their production AI agents by 2028 expected to pay 25% more for their liability insurance and have twice as many material audit findings. That's a forecast, not yet a market price. Treat it as a signal to plan for.

Where Do AI Observability and AI Evals Fit Into All of This?

Observability tells you what happened post-release, evals score it, and assurance proves it prior to release. You need all three, and they shouldn't have a single point of failure in common either.

The market is consolidating very rapidly. Dynatrace agreed in August to acquire Arize for $915m.

On 3rd November, CrowdStrike announced it will acquire Fiddler (to be sold as Falcon Observe) and Dreadnode (an AI red-teaming company), with CyberScoop, citing the Wall Street Journal, reporting the prices at approximately $400m and $95m respectively (CrowdStrike has not disclosed the exact figures).

That's good for the buyer - AI observability and LLM observability are becoming platform features. It does raise a new governance question, though.

As observability, evals and security all get folded into fewer and fewer platforms, Recuse applies to the stack too. Recuse applies at the org level as much as it does at the model level. That applies to every vendor, us included.

How Do You Introduce Model Diversity Into an Existing AI Governance Framework?

Add one control, one test suite, and one drill, and then make them release gates. You don't need a new framework at all; you just need the existing framework to fail a single model system.

A practical 90-day path:

• Inventory the critical outcomes, not the models. For each outcome, you have a producing model, a judging model, and a deprecation date for each. If the producer is the same as the judge, you're at Level 0 for that outcome.
• Write your behavioral tests at the outcome level. The tests that assert what must be true (the refund was in policy, the agent didn't call a tool it's not entitled to) are the ones that survive a model swap. Prompt-level snapshots aren't.
• Run the same suite against an alternative model on a monthly basis. Store the diffs. That turns Retire and Regress from surprises into tickets.
• Separate the checker. Wherever the outcome is deterministic, use deterministic assertions. Where judgment is required, use a model from a different family than the one that is the producer.
• Drill a revocation. Route a canary slice of a real workflow to the alternate (or run it in shadow mode). Time to recover and report per critical outcome.
• Gate releases on evidence. A new model version, prompt, or tool permission ships only when the suite passes on evidence that your team owns rather than on some vendor's dashboard. That's the ADLC equivalent of a CI gate.

Where TestMu AI Fits

TestMu AI's view is that quality engineering has become the independent verification layer for AI: it produces evidence, not opinions. Our products map to the three ways software is now being used.

For Human<->Software, we have KaneAI, which allows testers to write their tests in natural language, and Kane CLI, which gives the SDETs and developers the full QE cycle from the terminal. For Human<->Agent (chat, voice, IVR), we have Agent Testing, which tests the agents' conversations. For Agent<->Agent, we have our Agent Assurance offering (Rook), which has agents test agents.

HyperExecute orchestrates those suites in parallel so you can run them against a primary and an alternate model within a release cycle.

Our testing is deterministic in that the verdict comes from assertions you can read and re-run, not from the model's opinion of the output. If the AI is assisting in the authoring or exploration of the tests, then the verdict is still based on those inspectable assertions. That's what enables one suite to run against the primary and the alternate model and give you a diff that you can trust.

We hold ourselves to the same standard. A verification layer shouldn't be your next single point of failure. Ask any verification vendor (including ourselves) where the evidence is, whether it can be exported into your own systems and whether any of the verdicts are dependent on the model being tested.

What We Expect Next

These are predictions, grounded in the trends cited above:

• By the end of 2027, 'model concentration' will be a standard line item in the enterprise AI risk registers, just as 'cloud concentration' is. The FSB data and the Menlo numbers are already pointing in that direction.
• Deprecation notices become release gate triggers. With notice periods ranging from 60 days to six months, teams that do swap rehearsals will treat a deprecation as a sprint ticket; teams that don't will treat it as an incident.
• "Who checked the checker?" becomes a procurement question. As observability, evals and security all come together, the buyer will be asking for proof that the checker is independent of the model and the monitoring stack.

Conclusion: Govern Outcomes, Not Vendors

An AI governance framework that simply approves models without ever testing what happens when they change is governing the paperwork, not the risk. Two labs take up 86% of the enterprise LLM API spend.^[1] Models retire on a fixed notice period.

Research shows that models tend to prefer their own work. Add the 4R Continuity Test as a control, get all the key outcomes to Level 2 within two quarters (higher where the regulators are looking, of course) and make the evidence, not the model's opinion of itself, the condition of release.

That's the essence of AI trust really: you don't trust any one model more than any other; you just trust each one less.

Shipping agents that other agents use? Rook by TestMu AI does Agent Assurance: agents testing agents before release, with no human in the loop.

To run your first test, install Agent Assurance with npm install -g @testmuai/rook@latest command, or

book a demo

for a full walkthrough of the product.