Build Trustworthy AI Agents Powered by Evals [Testμ 2026]

The 100-Step Problem

Evals get written off as functional or unit testing with a new label, and his objection is about scale of consequence rather than technique. Agents are expected to perform long-running actions measured in hundreds of steps, not tens.

Every tool call and every shell command is a step in its own right. An error at step 20 has 80 more steps to propagate through, and nothing in the run flags that it happened.

His framing of the cost is practical. You spend time, money and debugging cycles on an outcome that is not what you expected, and sometimes you cannot even establish what went wrong.

The rest of the session is the toolkit for that, split between the challenges agents inherit from LLMs and the ones they create by having hands.

Defining an Eval

His definition has two halves that teams routinely reduce to one. An eval confirms the agent is doing what it is supposed to, and that it is not doing what it is not supposed to.

The organisational value is that it settles an argument nobody else can. Ship a new feature and the engineer, the quality engineer, the program manager, the product manager and the designer all need one source of truth on what good looks like.

Regression is the second reason. After the initial launch, iteration is where things stop working without anyone noticing, and the eval suite is what tells you that you have not broken what used to work.

The third reason is the one he flagged as his own unique take: portability. Not every team can spend heavily on proprietary models, and combining your own data with an open-source model and a fine-tuned checkpoint raises questions about where inference runs, on which cloud, on which GPUs. An eval suite lets you check the base capabilities hold regardless of any of that.

The Eval Harness

He built the picture in layers, starting from the model and ending at something you can run.

• The LLM - a next-token predictor, and a very good one, which on its own returns text rather than actions.
• Tools - search, browsing, access to your own data. Model plus tools is what produces agentic capability.
• Tasks - the specific things you want the agent to accomplish.
• Graders - deterministic code, an LLM acting as judge, or a human, depending on how much nuance the judgement needs.
• Measurement - cost and token usage tracked across all of it.

The collective trace of a run is what he calls the trajectory: the user prompt, the system prompt that is the model’s internal message to itself, and the reasoning traces, captured together as JSON.

Trajectories run against tasks and scored by graders are the eval harness. His aside on tokens is worth keeping: costs are subsidised in this window of AI history, and experimenting heavily now is a reasonable use of that.

Taming Non-Determinism

He was direct that there is no silver bullet, and framed it as a double-edged sword. The property that lets an LLM cover a much larger surface area is the same one that makes its behaviour hard to pin down.

• Temperature at zero - as deterministic as is theoretically available, and reproducible.
• Seeds - fixed as a hyperparameter, so a run can be repeated under the same configuration.
• Typed objects - emitted between passes to enforce the schema, so nothing unexpected slips between steps.
• Pass@k - out of five or fifteen attempts, how many times did the agent get it right.

His point about these is that they work around the problem rather than fix it. Running k times and counting the variation is how you learn what your actual reliability is.

Rewarding Abstention

His hallucination example is specific enough to reproduce. An agent needs to read a directory on the file system, it does not have permission, and the prompt told it not to stop until it had the answer.

Without access to the corpus, it invents. The prompt tweak that seemed like a good idea, picked up from somewhere on X, is what forced the invention.

The eval design that catches it is to reward abstention over confabulation. Construct the situation so the underlying corpus is inaccessible, then check what the agent does.

The correct behaviour is raising a hand and saying it lacks permission, so it cannot answer either way. Confident invention is the signal that more work is needed on adversarial probes.

Tool Call Safety

His model for this is that the LLM is the brain and the agent supplies the hands and limbs. Tool calls, whether MCP servers or plain APIs, are where actions become real, and where distributed systems knowledge earns its place.

Anything involving money is a critical write. Booking a flight or a hotel twice is a customer problem, not a test failure.

• Idempotency keys - reused across retries so a repeated call deduplicates rather than committing the action twice.
• Exponential backoff - waiting one, then two, then four, rather than polling a slow service every few seconds.
• Saga-style compensation - because undoing a wrong reservation is not deleting a database row, which in most cases sits in a back end you do not own. Cancellation has to be a first-class citizen so the fix is clean.
• Detailed failure classification - separating authentication and authorisation failures from 429 throttling from a step that produced malformed output.

He noted the asymmetry that makes this worth the effort. Agents read far more than they write, and the blast radius on a write is much larger.

Checkpoints and Rollback

His answer to the hundred-step problem is borrowed from systems engineering. Checkpoint every few steps, so a failure detected at step 25 can be traced back to a state captured at step 20 and resumed from there.

The requirement is that the persisted state be cohesive and self-sufficient. A checkpoint you cannot restart cleanly from is not a checkpoint.

With that in place, rolling back to the last known good state stops errors accumulating, and stops the run burning resources on work built on a bad foundation.

Memory Hygiene

He separated memory from context deliberately. Memory is what the agent bookmarks as it works, in files or a vector database; context is closer to RAM, the sliding window fed into the model for the next turn.

Even with a million-token window, not all memory reaches context, so past work has to stay retrievable or the agent runs in circles. What goes into memory needs sorting rather than accumulating.

• Facts against summaries - a stable fact can be locked down, where a summary has a finite lifetime because the base facts underneath it can change.
• TTLs - treated like a cache, keeping data accessible in the short run and expiring it afterwards.
• Provenance - the source, the time the inference was made, and a confidence level where the answer is not black and white.

His worked example for provenance involves an orchestrator running worker sub-agents. Worker three infers something from loose data at perhaps the fiftieth percentile of confidence while waiting on worker four, and once worker four returns the authoritative answer at P90, the low-confidence inference should be purged rather than left sitting in memory.

Chunking got the same treatment. Splitting by words is not enough for a coding agent, where chunking along the abstract syntax tree respects the structure the model actually needs.

His favourite failure to test for is stale memory. On a 50-person project with status updates arriving over months, an agent reading two-month-old updates produces a summary that misrepresents everyone’s work, and nothing about the output looks wrong.

Three Kinds of Grader

Grading splits into three buckets, each with its own strengths, and the goal is a strategy that covers the ground while spending the least money and keeping the most determinism.

• Code-based - fast, cheap, reproducible across runs, and sometimes as simple as a regular expression. Anything about tool calls suits them: valid parameters, correct JSON, schema conformance. He puts roughly 65 to 70 percent of what most teams measure in this bucket, and noted that frontier labs still lean on deterministic graders wherever they can.
• Model-based - for judgements code cannot make, such as whether writing is crisp, concise or citing authoritative sources. Non-deterministic by construction, since you are testing a model and its harness with another model, and in exchange you get flexibility across scenarios.
• Human - the gold standard, and expensive and slow. He recommends keeping graders and decision logs in version control, so a commit hash tells you exactly which criteria were applied.

Pairwise comparison is the model-based technique most people have already met without naming it. The tools that show two answers and ask which is better are running it with a human in the loop, and a judge model does the same against a rubric.

Building the rubric is a cross-functional exercise rather than an engineering one. Four or five dimensions, agreed between product, engineering and everyone else with a claim on what success means, then a curve across good behaviour, bad behaviour and what needs investigating.

He also gave a stopping rule for human grading, which teams rarely define. When the vast majority of human graders agree with what the agent did across a range of tasks, you have reached diminishing returns.

GAIA 2 and Mock Universes

He closed on work he collaborated on at Meta. GAIA 2 arrived in 2026, three years after the first version, and consists of around 800 carefully crafted scenarios.

Each scenario lives inside a mock universe modelled on an ordinary digital life: mail, messaging, a to-do list, shopping, apartment hunting. The apps are mock versions by necessity, because post-training and reinforcement learning runs cannot hit the real services.

His reasoning there is blunt. You would be blocked before you were even throttled, and you would control none of the variables that make the experiment meaningful.

Agents are evaluated across five dimensions: search, execution, adaptability, time and ambiguity. His example prompt shows why one instruction touches all of them.

Schedule a film production day, cancel any conflicts, email Ross because he is a required attendee, and reschedule if he declines or proposes another time. Scheduling is a straightforward calendar write; cancelling conflicts needs temporal awareness; there may be several people called Ross; and reacting to his counter-proposal is adaptability.

The results tell you where the field actually is. Search and execution score in the 80s and sometimes the high 90s, because that is where the last two or three years of effort went.

Ambiguity, adaptability and above all time remain hard for even frontier models. His reading of a poor temporal score is optimistic rather than damning, because it marks headroom you can still work through.

The scenarios are extensible, and Meta open-sourced the repository under the name agent research environments. In the resulting graph the original user prompt is one node and the oracle events are the rest, so each part of the agent’s behaviour can be validated as it goes.

Q & A Session

Time allowed for one audience question, and the answer was the most practical advice in the session.

• What is the most counterintuitive thing you have learned about your own agents?

  Rushabh Mehta: He started with hallucination having layers, where system prompts and agent instruction files are the low-hanging fruit. The genuinely counterintuitive part is that you have to read the raw traces, which cuts against the instinct that AI tooling is there to automate the workflow away. As an end user, or even as the developer of the agent, you mostly look at the final answer and any obvious API failures. What falls off silently is the interesting part, such as why the agent took path A when B and C were both available, and that is visible only in the full trace. He was firm that the first pass has to be manual, before any automation is built to read traces for you.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.