How to Build an Eval Harness for AI Agents

What Is an Eval Harness?

An eval harness is the code that runs an AI agent against a set of tasks, records each trial, grades the outcome, and aggregates the results into scores. For an agent that acts, it also sets up the environment each task starts from and inspects what the agent left behind.

The vocabulary below follows Anthropic's Demystifying evals for AI agents, and each term maps to one part of the harness built in this guide:

Two neighbors share the name. EleutherAI's lm-evaluation-harness scores model outputs on benchmark prompts, with no tools or environment, and an agent harness is the runtime code that turns a model into your agent. Agent harness vs eval harness compares the two in detail.

What Goes Into a Task File?

A goal, a fixture, and criteria that name their evidence. The task below asks a refund agent to refund a damaged order, seeds the order it needs, and lists three criteria the harness can check without trusting the agent's reply:

{
  "id": "refund-damaged-order",
  "goal": "Order A-1184 arrived broken. Refund it.",
  "fixture": { "orders.json": [{ "id": "A-1184", "total": 42, "status": "delivered" }] },
  "criteria": [
    { "id": "AC-1", "statement": "A refund row for A-1184 exists in the ledger", "check": "ledger_has", "orderId": "A-1184" },
    { "id": "AC-2", "statement": "issue_refund was called exactly once", "check": "tool_called_times", "tool": "issue_refund", "times": 1 },
    { "id": "AC-3", "statement": "The reply claims no refund the ledger does not show", "check": "claim_backed", "orderId": "A-1184" }
  ]
}

• Unambiguous criteria - Anthropic's test for a good task is one "where two domain experts would independently reach the same pass/fail verdict," so rewrite any criterion two reviewers could grade differently.
• Real fixtures - seed records the goal refers to, so a failure means the agent erred and not that the order was missing.
• A reference solution - keep a known-good run that passes every grader, which proves the task can be solved and the graders work.
• Should-not cases - add tasks where the right outcome is no action, such as a refund request above an approval limit.

Draw the first tasks from real failures and incidents, then grow the set with every new bug. The practice of writing the task before changing the agent is eval-driven development.

How Do You Keep Trials Isolated?

Give every trial a fresh environment and record exactly what it ran on. Anthropic's evals post says "Each trial should be isolated by starting from a clean environment," and describes Claude gaining an unfair advantage on some tasks "by examining the git history from previous trials."

• Fresh workspace - the harness below creates a temporary directory per trial, seeds the fixture, and deletes it afterward. For agents that run commands, use a container per trial instead.
• Per-sample sandboxes - the UK AI Security Institute's Inspect sandboxing docs state that "each sample gets its own sandbox instance," and their generated Compose configuration sets network_mode: none by default.
• Pinned resources - the infrastructure-noise study found infrastructure errors fell from 5.8% under strict resource enforcement to 0.5% when uncapped, and recommends treating resource configuration "as a first-class experimental variable."
• Recorded settings - write CPU, memory, timeouts, model ID, and the agent's commit into every report, so two runs can be compared honestly.

Note: TestMu AI Agent Assurance keeps run state as plain files in your project and records each run's version, profile revision, and scenario selection, so you can tell whether two runs are comparable. Create your TestMu AI account

How Does the Harness Invoke the Agent?

The same way a user or another system would. The harness below runs the agent as a command, passes the goal as an argument and the workspace as an environment variable, then collects three kinds of evidence: the reply, the trace the agent wrote, and the state of the workspace.

import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { spawnSync } from 'node:child_process';

const TRIALS = Number(process.env.TRIALS ?? 5);
const AGENT_CMD = ['node', [path.resolve('stub-agent.mjs')]]; // your agent's real command goes here

const readJson = (file) => (fs.existsSync(file) ? JSON.parse(fs.readFileSync(file, 'utf8')) : undefined);
const hasRefund = (ev, id) => (ev.ledger ?? []).some((r) => r.orderId === id);

function runTrial(task, trial) {
  const ws = fs.mkdtempSync(path.join(os.tmpdir(), task.id + '-')); // clean workspace per trial
  for (const [name, data] of Object.entries(task.fixture)) fs.writeFileSync(path.join(ws, name), JSON.stringify(data));

  const proc = spawnSync(AGENT_CMD[0], [...AGENT_CMD[1], task.goal], {
    env: { ...process.env, WORKSPACE: ws, TRIAL_SEED: String(trial * 7919 + 17) },
    encoding: 'utf8', timeout: 60_000,
  });
  if (proc.error || proc.status !== 0) return { trial, harnessError: proc.error?.message ?? 'exit ' + proc.status };

  const evidence = {
    claim: proc.stdout.trim(),
    ledger: readJson(path.join(ws, 'ledger.json')),
    trace: readJson(path.join(ws, 'trace.json')),
  };
  const verdicts = Object.fromEntries(task.criteria.map((c) => [c.id, graders[c.check](c, evidence)]));
  const v = Object.values(verdicts);
  const verdict = v.includes('fail') ? 'fail' : v.includes('unverified') ? 'unverified' : 'pass';
  fs.rmSync(ws, { recursive: true, force: true });
  return { trial, verdict, verdicts };
}

For this walkthrough, the agent is a stand-in script seeded per trial. It sometimes claims a refund it never wrote and sometimes skips writing its trace, so the harness has known failures to find. Replace AGENT_CMD with your agent's real command or an HTTP call to its endpoint.

// Demo stand-in for a real agent. Seeded per trial, it sometimes claims a refund
// it never wrote and sometimes runs without writing a trace, so the harness has
// known failures to find. Replace it with your agent's real command.
import fs from 'node:fs';
import path from 'node:path';

const ws = process.env.WORKSPACE;
let seed = Number(process.env.TRIAL_SEED);
const rand = () => ((seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648);

const writesTrace = rand() > 0.2;
const skipsRefund = rand() < 0.3;
const trace = [];

if (!skipsRefund) {
  fs.writeFileSync(path.join(ws, 'ledger.json'), JSON.stringify([{ orderId: 'A-1184', amount: 42 }]));
  trace.push({ tool: 'issue_refund', args: { orderId: 'A-1184', amount: 42 } });
}
if (writesTrace) fs.writeFileSync(path.join(ws, 'trace.json'), JSON.stringify(trace));
console.log('Your refund for A-1184 has been issued.');

A real agent usually exposes its trace through spans instead of a file. If you capture them with OpenTelemetry, note that the semantic conventions v1.42.0 release deprecated all gen_ai.* attributes, metrics, events, and spans in the main repository and moved them to a dedicated one, so pin your graders to the attribute names your SDK emits.

How Should Graders Decide Pass, Fail, or Unverified?

Each grader reads one piece of evidence and returns pass, fail, or unverified. Unverified means the evidence the grader needs is missing, which is a fact about the harness and the agent's observability, not a judgment on its work.

// Each grader returns 'pass', 'fail', or 'unverified' (the evidence it needs is missing).
const graders = {
  ledger_has: (c, ev) => (hasRefund(ev, c.orderId) ? 'pass' : 'fail'),
  tool_called_times: (c, ev) => {
    if (!ev.trace) return 'unverified'; // no trace, so the harness cannot see tool calls
    return ev.trace.filter((t) => t.tool === c.tool).length === c.times ? 'pass' : 'fail';
  },
  claim_backed: (c, ev) => (/issued|refunded/i.test(ev.claim) && !hasRefund(ev, c.orderId) ? 'fail' : 'pass'),
};

• Grade the outcome - Anthropic's guidance is that "it's often better to grade what the agent produced, not the path it took," so ledger_has reads the ledger and ignores the order of calls.
• Check claims against state - claim_backed fails any reply that reports a refund the ledger doesn't show, which is the false success a reply-only grader passes.
• Trial verdict rules - any verified failure fails the trial, unverified evidence with no failure makes the trial unverified, and only all-pass trials pass.
• Model graders, calibrated - use a model judge only for qualities code can't check, and measure it against human labels first. In the iOSWorld benchmark paper, task-level judgments agreed with human annotators 89% of the time (κ=0.77) across 128 trajectories.

Watch for tasks that never pass. Anthropic notes that "a 0% pass rate across many trials (i.e. 0% pass@100) is most often a signal of a broken task, not an incapable agent," so read those transcripts before blaming the agent. The guide to LLM-as-a-judge covers rubrics and calibration for model graders in more depth.

How Many Trials Should You Run per Task?

Enough to measure consistency, because an agent that passes once in three tries isn't one you can ship. Report two numbers per task: pass@k, the chance at least one of k trials passes, and pass^k, the chance all k pass. The main loop computes both from the decided trials and leaves unverified trials out of the denominator:

const comb = (n, k) => (k > n ? 0 : [...Array(k)].reduce((a, _, i) => (a * (n - i)) / (i + 1), 1));
const passAtK = (n, c, k) => 1 - comb(n - c, k) / comb(n, k); // at least one of k trials passes
const passHatK = (n, c, k) => comb(c, k) / comb(n, k);       // all k trials pass

const report = { tasks: [] };
for (const file of fs.readdirSync('tasks')) {
  const task = JSON.parse(fs.readFileSync(path.join('tasks', file), 'utf8'));
  const trials = Array.from({ length: TRIALS }, (_, i) => runTrial(task, i + 1));
  const decided = trials.filter((t) => t.verdict === 'pass' || t.verdict === 'fail');
  const passed = decided.filter((t) => t.verdict === 'pass').length;
  const cells = trials.flatMap((t) => Object.values(t.verdicts ?? {}));

  for (const t of trials) console.log('trial ' + t.trial, t.harnessError ? 'HARNESS ERROR ' + t.harnessError : t.verdict.padEnd(10) + ' ' + Object.entries(t.verdicts).map(([k, v]) => k + ' ' + v).join('  '));
  const summary = {
    id: task.id, trials: TRIALS, harnessErrors: trials.filter((t) => t.harnessError).length,
    decided: decided.length, passed, unverifiedTrials: trials.filter((t) => t.verdict === 'unverified').length,
    unverifiedCriteria: cells.filter((v) => v === 'unverified').length + '/' + cells.length,
    'pass@1': passAtK(decided.length, passed, 1).toFixed(2),
    'pass^3': passHatK(decided.length, passed, 3).toFixed(2),
  };
  console.log(summary);
  report.tasks.push({ ...summary, trials });
}
fs.writeFileSync('report.json', JSON.stringify(report, null, 2));
// Exit 0 means the harness ran. Whether the agent passed is in report.json.

Running it with TRIALS=8 node harness.mjs against the stand-in agent produced this output:

trial 1 fail       AC-1 fail  AC-2 unverified  AC-3 fail
trial 2 pass       AC-1 pass  AC-2 pass  AC-3 pass
trial 3 fail       AC-1 fail  AC-2 fail  AC-3 fail
trial 4 pass       AC-1 pass  AC-2 pass  AC-3 pass
trial 5 fail       AC-1 fail  AC-2 unverified  AC-3 fail
trial 6 pass       AC-1 pass  AC-2 pass  AC-3 pass
trial 7 fail       AC-1 fail  AC-2 fail  AC-3 fail
trial 8 unverified AC-1 pass  AC-2 unverified  AC-3 pass
{
  id: 'refund-damaged-order',
  trials: 8,
  harnessErrors: 0,
  decided: 7,
  passed: 3,
  unverifiedTrials: 1,
  unverifiedCriteria: '3/24',
  'pass@1': '0.43',
  'pass^3': '0.03'
}

Four of the eight replies claimed a refund the ledger never got, and AC-3 failed on every one of them. Trial 8 shows the third verdict at work: the refund landed, but no trace was written, so AC-2 couldn't be checked and the trial stayed out of the pass rate instead of counting as a pass. A pass@1 of 0.43 next to a pass^3 of 0.03 is the gap explained in pass@k vs pass^k.

How Do You Gate CI on Eval Results?

Split running from judging. The harness exits 0 whenever it ran, and a separate gate script reads report.json and decides whether the build may continue:

import fs from 'node:fs';

const { tasks } = JSON.parse(fs.readFileSync('report.json', 'utf8'));
let blocked = false;
for (const t of tasks) {
  if (t.harnessErrors > 0) { console.log(t.id + ': ' + t.harnessErrors + ' trial(s) never ran, fix the harness first'); blocked = true; }
  if (Number(t['pass^3']) < 0.8) { console.log(t.id + ': pass^3 ' + t['pass^3'] + ' is below 0.80'); blocked = true; }
  console.log(t.id + ': unverified criteria ' + t.unverifiedCriteria + ' (reported, not counted as pass or fail)');
}
process.exit(blocked ? 1 : 0);

On the demo report, the harness exited 0 and the gate exited 1 with this message:

refund-damaged-order: pass^3 0.03 is below 0.80
refund-damaged-order: unverified criteria 3/24 (reported, not counted as pass or fail)

• Harness errors first - a trial that never ran says nothing about the agent, so the gate reports it separately and asks for a harness fix.
• Thresholds you've earned - measure pass^k and the unverified count for a few weeks before you set a blocking threshold on either.
• Small gaps - the infrastructure-noise study concludes that "leaderboard differences below 3 percentage points deserve skepticism," and the same applies when comparing two of your own runs.

When Should You Use Agent Assurance Instead of Building Your Own?

When your agent acts on systems and you'd rather not maintain task generation, invocation, and graders yourself. TestMu AI Agent Assurance tests how your agents actually behave across workflows, tools, and actions, and catches failures and vulnerabilities before they ship. Each piece of the harness above has a counterpart:

Scenarios are plain YAML files you can review in a pull request, and the Agent Assurance scenario docs describe each field, from acceptance criteria to the evidence a check requires. Agent Assurance is pre-alpha, so commands and stored file formats can still change, and it runs your agent for real, so point it at staging. Keep a custom harness for domain scoring that no general tool knows, such as a pricing model's tolerances.

Next Steps

Start your eval harness with one task: turn your most recent agent incident into a task file with a fixture that reproduces it and a criterion that reads the record it got wrong. Run it eight times in fresh workspaces and look at pass^3 and the unverified count before you add a second task.

To skip building the harness, install Agent Assurance with npm install -g @testmuai/rook and run its generated scenarios against your staging agent.