Agent-as-a-Judge: How Agent Judges Grade What AI Agents Did

What Is Agent-as-a-Judge?

Agent-as-a-judge is a way to evaluate an AI agent with another AI agent. The judge receives the task's requirements, the workspace the agent produced and, in some setups, the agent's step-by-step trajectory, and it uses tools to find and read the evidence for each requirement before it decides whether that requirement was met.

The term comes from Agent-as-a-Judge: Evaluate Agents with Agents by Mingchen Zhuge and 12 co-authors, first posted to arXiv on 14 October 2024, which calls it "an organic extension of the LLM-as-a-Judge framework, incorporating agentic features that enable intermediate feedback for the entire task-solving process". The authors tested it on code generation only, with DevAI, a benchmark they built of 55 AI development tasks and "a total of 365 hierarchical user requirements".

A revised version appeared in the ICML 2025 proceedings (PMLR volume 267), and its abstract keeps the same scope: "55 realistic AI code generation tasks" with 365 solution requirements. Earlier code benchmarks such as the HumanEval benchmark need no judge at all, because they score a model by running its completions against unit tests.

A January 2026 survey, Agent-as-a-Judge by Runyang You and seven co-authors, explains the shift to agentic judges by the limits of single-call ones. Their reliability, it says, "has become constrained by inherent biases, shallow single-pass reasoning, and the inability to verify assessments against real-world observations".

The related terms overlap but are not interchangeable:

• Agent judge - the agent doing the grading in an agent-as-a-judge setup, with access to the workspace and tools.
• Agentic judge - the survey's umbrella term for judges that plan, verify with tools, work with other agents or keep memory between judgments.
• Judge agent - in LangWatch's Scenario testing framework, an LLM-powered evaluator that reads a simulated conversation after every turn and decides whether the test continues, succeeds or fails.
• Gray-box and black-box judging - in the DevAI study, a gray-box judge also reads the developer agent's trajectory, while a black-box judge sees only the workspace the agent left behind.

How Is Agent-as-a-Judge Different From LLM-as-a-Judge?

An LLM judge makes one model call over material someone else selected and placed in its prompt. An agent judge chooses what to look at, which matters when the evidence is spread across a folder of files and a long run log.

The agreement figures come from Table 3 of the Agent-as-a-Judge paper's full text, across MetaGPT, GPT-Pilot and OpenHands, and the times from its cost analysis. The LLM judge's best black-box figure, 84.15% on MetaGPT, says little: the authors note that "since MetaGPT rarely meets requirements, LLM-as-a-Judge easily identifies most cases as negative". The same table notes that gray-box runs "use carefully manually collected trajectory data", which it calls "nearly inaccessible in practical situations".

Against people, the paper's PDF reports that the three human experts, judging alone, matched their own later consensus on 76.23% to 92.63% of requirements, so the gray-box agent judge sits inside the range of a single expert. The human evaluation, debate round included, took the experts a self-reported 86.5 hours, while the agent judge needed 118.43 minutes and 30.58 USD in API calls against the 1,297.50 USD the authors estimate for the experts at 15 USD an hour: about 2.28% of the time and 2.36% of the cost. The paper's text gives 2.29% for cost and 2.36% for time, the other way round.

For a judge that only reads replies and transcripts, the LLM-as-a-judge guide covers rubric design, scoring methods and the biases to correct for. G-Eval is a metric of that kind: a judge model scores text against plain-language criteria, follows evaluation steps it generated itself and weights each rating by its token probability.

What Does an Agent Judge Do That a Single Model Call Cannot?

It goes and gets evidence. The DevAI judge was assembled from separate modules, and the best-performing version kept these:

• Graph - maps the project's files, modules and dependencies so the judge knows what exists before it reads anything.
• Locate - finds the folder or file a requirement refers to, such as src/data_loader.py.
• Read - opens files in many formats, code, images and documents included, so a requirement about a saved plot can be checked against the plot.
• Retrieve - pulls the relevant steps out of a long trajectory, which only a gray-box judge has.
• Ask - decides whether the requirement is satisfied, given everything the other modules gathered.

The search, memory and planning modules were dropped. Memory made results worse because an early wrong judgment fed later ones, and planning proved unstable.

How Closely Do Agent Judges Agree With Human Judges on the DevAI Coding Benchmark?

The DevAI authors released every human consensus verdict and every gray-box agent judge verdict as JSON files, so the paper's agreement figures can be checked without an API key. A recount run on 2026-10-19 reproduces them and adds two things the paper's table does not show: the direction of each disagreement and how much agreement a judge that never looks at the work would get.

// aaj-core.cjs: human vs agent-judge verdicts on DevAI
// node aaj-core.cjs  (Node 18+, no API key)
const repo = "metauto-ai/agent-as-a-judge";
const raw = "https://raw.githubusercontent.com/"
  + repo + "/main/";
const api = `https://api.github.com/repos/${repo}`;
const devs = ["MetaGPT", "GPT-Pilot", "OpenHands"];
const get = async (u) => (await fetch(u)).json();

(async () => {
  const tree = (await get(`${api}/git/trees/main`
    + "?recursive=1")).tree.map((x) => x.path);
  console.log(
    "Agent      Reqs   Agree  Human  Judge  FM  FN");
  for (const dev of devs) {
    const dir = `benchmark/judgment/${dev}/`;
    const files = tree.filter((p) => p.startsWith(
      dir + "human_as_a_judge/") && p.endsWith(".json"));
    let n = 0, ok = 0, fm = 0, fn = 0;
    let met = 0, said = 0;
    for (const hp of files) {
      const name = hp.split("/").pop();
      const ap = dir + "agent_as_a_judge/gray_box/"
        + name;
      if (!tree.includes(ap)) continue;
      const H = await get(raw + hp);
      const A = await get(raw + ap);
      const judge = new Map(A.requirements.map(
        (r) => [r.requirement_id, r.satisfied === true]));
      for (const r of H.requirements) {
        if (!judge.has(r.requirement_id)) continue;
        const a = judge.get(r.requirement_id);
        const h = r.satisfied === true;
        n++; met += h; said += a;
        if (a === h) ok++;
        else if (a) fm++;
        else fn++;
      }
    }
    const p = (x, d = 1) =>
      (100 * x / n).toFixed(d) + "%";
    console.log(dev.padEnd(10), String(n).padStart(4),
      p(ok, 2).padStart(7), p(met).padStart(6),
      p(said).padStart(6), String(fm).padStart(3),
      String(fn).padStart(3));
  }
  console.log("Human, Judge = share of requirements met");
  console.log("FM = judge met, humans not met");
  console.log("FN = judge not met, humans met");
})();

Agent      Reqs   Agree  Human  Judge  FM  FN
MetaGPT     366  92.08%  22.1%  23.5%  17  12
GPT-Pilot   366  86.61%  44.5%  46.4%  28  21
OpenHands   366  90.16%  42.9%  43.4%  19  17
Human, Judge = share of requirements met
FM = judge met, humans not met
FN = judge not met, humans met

Chance check: agreement of a judge that always says
'not met', and Cohen's kappa of the agent judge
  MetaGPT   always-not-met 77.87%  kappa 0.78
  GPT-Pilot always-not-met 55.46%  kappa 0.73
  OpenHands always-not-met 57.10%  kappa 0.80

Judge 'met' verdicts that humans also marked met
  MetaGPT    69 of  86  80.23%
  GPT-Pilot 142 of 170  83.53%
  OpenHands 140 of 159  88.05%

Do Agent Judges Err Toward False Passes or False Failures?

On DevAI, mostly false passes, which are an agent judge's false positives. In the recount the judge said met where humans said not met 17, 28 and 19 times for MetaGPT, GPT-Pilot and OpenHands, against 12, 21 and 17 errors in the other direction, or 64 against 50 overall.

A false met costs more than a false not met:

• False not met - sends a person to look at work that was fine, which costs review time and nothing else.
• False met - lets unfinished work through, and in an agent pipeline it often repeats the agent's own claim that the work was done, the failure the guide to agent action hallucination traces through agent runs.
• Gate on false met - when an agent judge decides releases, tune and audit it on its met verdicts first, because those are the errors that reach users.

Disagreements by requirement category, all agents
Category                      Reqs  Agree  FM  FN
Visualization                  198    91%  13   4
Machine Learning Method        186    91%  10   6
Data pre/postprocessing        183    89%   6  14
Dataset or Environment         180    87%  13  11
Other                          129    88%   9   7
Performance Metrics            105    93%   4   3
Human Computer Interaction      75    89%   4   4
Save Trained Model              39    90%   3   1
(no category)                    3    33%   2   0

By prerequisites (all agents)
  no prerequisite     297 reqs  88.22%  FM 17  FN 18
  has prerequisite    801 reqs  90.14%  FM 47  FN 32

FM = judge said met, humans said not met
FN = judge said not met, humans said met

Why Must an Agent Judge Use Read-Only Tools?

Because checking with a write tool performs the action being checked. A judge that calls a ticket tool to see whether the ticket exists files a new ticket, and the verdict then describes the judge's work instead of the agent's.

The 2026 agentic judge survey lists safety among the four open challenges for the field: "Tool-augmented judges may access external systems such as search engines, code executors, or databases, which expands the attack surface for prompt injection, tool misuse, or unintended side effects."

• Read paths only - give the judge file reads, status endpoints, trace queries and read-only database credentials, never the write tools the agent under test holds.
• Untrusted input - the files and records a judge reads were produced by the agent under test, so treat any instructions inside them as data.
• Writes as unverified - when only a write could confirm a result, record the requirement as unverified instead of letting the judge act.

How to grade an agent on the traces and side effects it left, and how to test the checker itself, are covered in the guide to the verification agent, in its sections on grading an agent on evidence it left behind and on verifying the verifier.

What Should an Agent Judge Return When It Cannot See Evidence for a Requirement?

A third verdict, separate from met and not met. DevAI's judgment files allow only true or false, so every requirement the judge could not check was still forced into met or not met in the recount above.

That third verdict is most useful when it names what is missing, such as the file, record or log the judge needed, so the gap can be closed. SimpleQA, a factuality benchmark, scores a comparable third outcome: each reply is marked correct, incorrect or not attempted, so an abstention costs less than a wrong guess.

The guide to the test oracle problem covers how to report results nobody could verify. The Agent Assurance results and evidence guide defines Unable to Verify and reads pass rate and coverage apart: its worked example shows a 100% scenario pass rate on a run where 12 of 20 acceptance criteria ended Unable to Verify, a criterion coverage of 40%.

Limits of the Published Agent-as-a-Judge Research

The measured case for agent-as-a-judge rests on one benchmark, and the survey's PDF version adds the running costs: agentic evaluation "typically involves multiple reasoning steps, intermediate decisions, and coordination among multiple agents, all of which increase computation per evaluation."

• Code generation only - every agreement figure above comes from 55 AI development tasks with small workspaces, so it does not transfer to support, browsing or workflow agents without a new comparison against human labels. The GDPval benchmark has one such comparison for work deliverables: its paper checks an automated LLM grader against blinded occupation experts.
• The consensus is not ground truth - the DevAI authors say some error likely remains in their human consensus even after the experts debated each requirement.
• Gray-box data is hard to get - the strongest results used trajectories collected by hand, which a production agent rarely provides.
• Latency - the agent judge is the slower of the two, as the timing row of the comparison table shows, too slow for feedback loops that need an answer in seconds.
• Model drift - the judge-model swap described above moved agreement with no other change, so a judge needs re-checking against human labels whenever its model changes.

How Agent Assurance Uses Judges for Agents That Act

The DevAI judge graded code sitting in a folder. An agent that acts changes tickets, records and files in other systems, so its judge needs the same habits on a wider surface: one requirement at a time, evidence it collects itself and no writes of its own.

TestMu AI's Agent Assurance applies that design to agents you own. The product is pre-alpha and publicly installable; you add its command line, Rook CLI, from npm with one command:

npm install -g @testmuai/rook

You can run it on macOS and Linux, and 64-bit Windows through npm or WSL. For the npm install, have Node.js 20 or newer on the machine.

• A judge per criterion - scenarios are generated from your agent's code or spec with acceptance criteria, and a judging subagent rules on each criterion separately, the same unit DevAI's requirements use.
• Evidence the judge collects - the judge opens the files the run changed inside the paths your profile lists and any artifacts it left, lines every tool call up against the agent's declared tool list (the never-call entries too), and looks records up with a read-only MCP tool you have approved.
• Mixed checks - a model judge scores some criteria, while a read of the changed system or a rule such as a regex scores others, and in every case the agent's own claim that it acted is not accepted as proof.
• Read-only by instruction - judges are told to verify without changing anything, and each judge tool call passes the same permission gate as any other; the Rook CLI permissions and safety docs ask for a read endpoint, trace, file observation or read-only MCP tool, and read-only credentials are yours to supply.
• A third verdict - a criterion the judge cannot settle from evidence is reported as Unable to Verify, next to Pass and Fail, and the pass rate leaves it out.

Most eval and observability tools grade what an agent said and what its trace recorded; Agent Assurance looks at what the run changed and reports separately what it could not verify. It calls your actual agent before release, from a terminal or a CI job, and because the agent's writes during a run are real, point it at staging. It does not monitor production and is not a runtime guardrail.

Adding an Agent Judge to Your Own Agent Evaluations

Measure an agent-as-a-judge setup against people before you let it decide anything:

• Take requirements from tasks your agent already runs and have at least two people label each one met or not met, then settle their disagreements together, as a small golden dataset.
• Run your agent judge on the same requirements with read access to the workspace and the run's logs.
• Report agreement beside the always-not-met baseline, the false met and false not met counts, and the share of met verdicts people confirmed, the columns the DevAI recount produced.

A single labeller makes a weak reference, as the DevAI study shows. In the first arXiv version of the Agent-as-a-Judge paper, the DevAI expert who erred most differed from the final consensus on 23.77% of GPT-Pilot's requirements, and a majority vote of all three experts corrected most of those errors.

If the agent writes to files, tickets or databases, give the judge read-only access and a third verdict for missing evidence, or follow the Agent Assurance quickstart to install Rook CLI and see per-criterion verdicts with their evidence on an agent you own.