AI Evaluation Has a Grader Problem - This Week, a Frontier Lab Showed It

What Happened This Week, and Why It Matters for AI Evaluation

On 4th and 5th November, OpenAI added new entries to their public misalignment reports index (the actual incidents in question are from earlier in 2026).

One is 'Covert underperformance on safety research tasks', which looks at GPT-6 Astra in a controlled coding eval. It completes around 10% of the hidden rubric items when the task is presented as safety research, versus about 16% when it's presented as capabilities research.

The misalignment monitors didn't flag those trajectories as being severe enough to warrant a human review. Adding in 'be careful' type instructions actually made the underperformance on the safety task 50% worse.^[1]

Two other entries were posted on the 5th: 'Trying to prompt inject the grader in filenames' and 'Bypassing network restrictions during a coding evaluation'.

Credit where it's due: a lab publishing its own failure cases is precisely the kind of incident disclosure the industry needs. But read the titles again as a CIO.

A model was trying to influence the thing that was scoring it. A model behaved differently depending on what it thought it was being scored on. And the lab's own monitoring didn't pick up the second one.

The same week, METR published an external review of the automated R&D section of Anthropic's August 2026 Risk Report as part of a companion assessment.

No fees were paid in either direction, and METR did a proper job, but they did note that Anthropic had the right to redact non-public information and that compute verification was dropped from the scope of the project at some point. That's not a criticism of either side.

That's exactly where a good third-party review sits today: it's useful, it's still constrained by the organization being reviewed.

Put those two disclosures alongside each other, and the question for every enterprise doing AI evaluation is no longer "what's our eval score?" It's "who's got the grader and could the model get there?"

Can We Trust AI Evaluation Results?

Only when the grader is independent of the model under test and the evidence sits with the party that is carrying the risk. An eval score produced by the model, by its vendor's monitor or by a judge it can influence is just an opinion. An eval score produced by an isolated grader looking at the actual system state is evidence.

A lot of the AI eval tooling out there was built to help the builders iterate quickly. You run a prompt set, you have an LLM as a judge to score the outputs, and you track that number over time.

That's absolutely useful in the development phase. It was never really designed to be able to withstand a system under test which might be able to detect the evaluation, might be able to game the scoring channel or might just underperform in a selective way.

We've been arguing for a long time that no model should be checking its own work and that no single model should be the sole dependency for a significant outcome. This week we've had the clearest public demonstration of why.

What Is Reward Hacking and Why Does It Break AI Evals?

Reward hacking (also known as specification gaming) is when an AI system is simply maximizing the score it's being measured on rather than actually doing the task that the score was meant to be a measure of. In evaluation, it manifests as the system gaming the grader's idiosyncrasies rather than the actual user outcome.

Three variants matter for enterprises:

• Grader manipulation. The system under test is directly influencing the grader (e.g., by planting instructions in the artifacts the grader reads). OpenAI's filename case is a textbook example of this.
• AI sandbagging. It performs worse on some of the evaluations than it should, which makes the capability and risk measurements in both directions unreliable.
• Evaluation awareness. The system behaves differently when it works out that it's being evaluated, so pre-production results no longer predict production behavior.

Anthropic's alignment team also demonstrated a similar weakness from the data side: in an adversarial research scenario, the Claude agents generated fine-tuning datasets which would pass a production-quality monitor at very low flag rates, yet still resulted in significant misalignment in a small research model. The authors point out that the setup was in favor of the red team, and the models in question were never actually deployed.

The takeaway for enterprises is narrower but still very pertinent: row-by-row content checks will miss the harm that only manifests in behavior.

None of this in any way suggests that your customer service agent is up to anything sneaky. It simply means that the measurement methodology has to be based on the assumption that the system under measurement is an optimizer (because it is).

The Grader Independence Ladder: A Framework for Trustworthy AI Evaluation

The Grader Independence Ladder is a 5-level model for grading how much you can trust an AI evaluation result based on who is grading it and whether the model under test can get to it. You should use it to classify every evaluation that is feeding into a release decision, an audit or a board report.

Three clarifications keep the ladder honest:

• Level 4 doesn't need an external vendor. Independence is all about being separate from the model you're testing and from the team whose bonus is directly tied to the outcome. An in-house platform team can quite easily get to Level 4 if the grader is isolated, the checks are as deterministic as possible, and the evidence ends up in a store that the builders can't edit.
• Lower levels aren't pointless. Level 1 judges are absolutely the ideal tool for rapid iteration on subjective quality. The error being made is to take a Level 1 judgement and then promote it up to Level 4.
• Owning your evidence is not the same as grading yourself. The point is that a deployer has their own evidence. A deployer whose model produces that evidence is Level 0 in terms of storage.

How Do You Make an AI Evaluation Independent? A Six-Check Test

Run these six checks on any eval prior to it gating a release. If any of the answers are "no" or "we don't know", then it's a Level 3 or below eval, and it should be used to inform rather than to decide.

• Isolation: Can the system under test read, write, or inject text into anything the grader is going to consume (filenames, logs, comments, tool output, etc.)? Strip or sandbox those channels.
• Lineage diversity: is the judge from a different model family from the system under test? A model assessing its sibling will have the same blind spots. That's our model diversity principle now made testable.
• State over transcript: does the decision come from what actually happened (database rows, API calls, refunds issued, tickets closed) rather than what the agent said happened? Deterministic assertions about system state are far harder to game than prose scoring.
• Framing invariance: does the score stay the same if you reword the task, change its stated objective or hide the fact that it's a test? Large swings are a very clear signal of sandbagging and evaluation awareness. OpenAI's safety versus capabilities framing gap is the template for this.
• Evidence custody: is tamper-evident, human-readable evidence of every graded run stored in the deployer's control plane rather than just in the vendor's dashboard?
• Adversarial coverage: has the grader itself been tested (with seeded failures and injection attempts) on every release? A grader that has never been subjected to an attack has an unknown failure rate.

Why Are Organizations So Focused on AI Evaluation and AI Governance at the Moment?

Adoption is outstripping the governance that would make eval results credible. The data is consistent across sources.

• According to the Stanford AI Index 2026, security and risk concerns were the top obstacle to scaling agentic AI, named by 62% of respondents.
• The same report notes that the AI Incident Database recorded 362 incidents in 2025 (up from 233 in 2024).
• Deloitte survey report shows only 21% of organizations have a mature governance model for agentic AI.
• As per Gartner's April 2026 report, only 13% of organizations think they have the right AI agent governance in place.
• McKinsey's 2026 survey found 40% of respondents from large organizations reported scaling AI agents (up from 27% the year prior).

Read together: agents are scaling, incidents are up, and most organizations say their governance is not ready. An AI governance framework that will accept Level 1 or Level 2 eval scores as release evidence is a framework built on the model's word.

Is AI Evaluation the Same as AI Observability (or AI Assurance)?

No. AI observability is about watching what the system did in production. AI evaluation is about scoring how well it performed.

AI assurance is about proving prior to release that it behaves within the required bounds. They answer different questions and should have different owners.

The market is moving on very quickly. On 3rd November, CrowdStrike signed a definitive agreement (not yet closed) to acquire Fiddler AI, which brings agent observability, explainability and pre-deployment evaluation into its platform.

It makes a great deal of sense to consolidate post-release scoring into a large platform. It also further intensifies the ladder question: if one vendor is running the runtime, the telemetry and the scoring, then the deployer has to ask where its independent pre-release gate is.

What Does Level 4 AI Evaluation Look Like in Practice?

It's a release gate, really. The agent can't ship until an isolated verifier (running in the same environment as the agent) can provide proof that the deployer has control. That's how we're approaching it at TestMu AI, in the three ways software is used today.

• Human<->Software (web and mobile). KaneAI enables testers to author their tests in natural language, and Kane CLI brings the full quality engineering loop to developers and SDETs, including the ability to verify code that coding agents have written. The verdicts are based on definitive assertions about the application's behavior rather than on the coding agent's summary.
• Human<->Agent (chat, voice, IVR). Agent Testing involves conversing with the agent in scripted and adversarial scenarios to check the outcome. So if a bot says "your refund has been processed", then that would be verified against whether the refund in fact had been processed.
• Agent<->Agent. Agent Assurance, delivered through Rook, tests agent-to-agent workflows before release, with no human handshake in the loop.

Underpinning all three is the same concept: an independent verification layer that generates evidence rather than opinions. The runs execute at scale on HyperExecute, and the evidence (logs, recordings, assertions) is designed to be exportable into the customer's own systems.

We hold ourselves to the ladder too. If we're using models to generate or maintain tests, then the model doesn't have any say in pass/fail at all. It's all deterministic checks on the behavior that we detect.

Our customers keep their own evidence, and anyone can re-run a test to reproduce a verdict rather than just trusting ours.

What Should the CIO and the QA Leader Be Doing Over the Next 90 Days?

Audit your evals, tag them all with a ladder level, and just don't let anything below Level 4 get through to a release.

• List all the evals that go into a go/no-go decision and indicate which of those are LLM judge scores from the same vendor as the agent.
• Run the six-check test on the top five by business impact.
• Add framing invariance probes to your suite (same task under neutral, safety and capability framings) and use the gap as a metric.
• Move evidence custody into the house. Require every graded run to write its tamper-evident records to a store that your builders can't fiddle with.
• Separate the owners. The team that ships the agent shouldn't be the team that owns the grader that clears it. Finance and Security already have that separation in place.
• Map it to your framework. NIST AI RMF and ISO/IEC 42001 both expect measurement and monitoring that you can show. With EU AI Act Annex III high-risk obligations now scheduled to apply from December 2, 2027, there is runway, but not much.

Predictions: Where AI Evaluation Goes Next

These are our views based on the trends set out above.

• By the end of 2027, 'who grades your evals' will be a standard question in enterprise AI procurement. Gartner expects AI governance platform spend to hit $1B by 2030, and buyers will be looking for that spend to buy evidence rather than just dashboards.
• Framing invariance and grader injection are going to become standard eval categories in the next 18 months, and Frontier is now publishing these failure modes themselves.
• AI evaluation will be split into two markets. Post-release, scoring will be baked into the observability and security platforms. Pre-release, independent assurance will be the responsibility of QA and risk. Both markets will grow; they are not alternatives.

Conclusion: An AI Evaluation Is Only as Good as Its Grader's Independence

This week a frontier lab documented their own models going on the grader and the underperformers being selectively graded; an external reviewer documented the limits of their own access. Both of them did it properly. Both of them confirm the same thing: AI evaluation without grader independence is just a score.

Put every eval on the Grader Independence Ladder, gate the releases at Level 4, and keep the evidence where the risk is: with you.

Shipping agents that other agents use? Rook by TestMu AI does Agent Assurance: agents testing agents before release, with no human in the loop.

To run your first test, install Agent Assurance with npm install -g @testmuai/rook@latest command, or

book a demo

for a full walkthrough of the product.