Why RL Environments Are All You Need [Testμ 2026]

Superhuman and Failing

Agents are solving decades-old mathematical conjectures. Coding agents write tens of thousands of lines. Deployed into production, the revenue impact does not follow.

He noted that some organisations have started talking about pulling agents back out of production, which makes the question worth asking properly rather than rhetorically.

His diagnosis is reliability, and specifically reliability on your task rather than in general. The agent is not unreliable; it is unreliable at the thing you built it for.

The Reliability Gap

Frontier models are trained on a specific set of data and environments, curated for writing poetry, writing code, doing taxes and whatever else the labs decide to cover. Your production tasks are somewhere else entirely.

Unless you ship your tasks to the labs, which most enterprises have no intention of doing, they cannot train on your distribution.

What the labs do instead is collect more and more data to encompass more of the space, which means larger and larger models. He drew the commercial consequence out plainly: token prices go up, and buyers are already seeing model prices rise in exchange for reliability.

ELIZA and the AI Winter

His historical detour is short and does real work. In the 1960s, expert systems encoded intelligence as handwritten if-then rules, and people believed general intelligence was close.

ELIZA looked convincing until it became obvious it was following a script and did not generalise. Confidence collapsed, investment followed it down, and the AI winter set in.

Deep learning ended it by replacing handcrafted features with models that learn from data, which is the bitter lesson in one line: methods that leverage compute generalise far better than handcrafted methods built on human intuition.

The recipe that follows has been stable for fifteen years. Curate data for the task you have in mind, apply compute to a model, and it learns your distribution.

From Prompts to Environments

Applying that recipe to agents means asking what agent data actually is, and his answer is that it changes shape entirely.

For chatbots the data was supervised fine-tuning pairs: a prompt and a response, a question and how to answer it. That suited a technology being asked for knowledge.

Agents act rather than answer. Open a coding agent and it has an environment, meaning your laptop and a particular folder, and it takes actions inside it.

So agent data is environments and verifiers. Where traditional machine learning had a label, an agent needs a verifier that decides whether the task set in the environment was actually solved.

Inside an RL Environment

An RL environment captures the world the agent acts in, built as a contained replica of your production system so you can train and evaluate inside it.

His concrete example is a set of databases and dbt transformations mimicking an e-commerce store, with a task attached: calculate the revenue for Q1 of 2026.

The agent is dropped in with access to the files, the databases and a specific set of tools. It runs queries, does some maths, writes a script, and returns an answer.

What it did along the way often matters less than the verifier’s judgement on whether it got there. Standard tooling and formats for describing these environments are beginning to emerge.

Model, Harness, Prompt

An agent, reduced to its parts, is an LLM wrapped in a harness and given a goal. Those three components are what there is to optimise.

In practice most teams freeze two of them. The model is whatever frontier release you picked, the harness is whatever coding agent you use, and all the iteration lands on the prompt.

The bitter lesson says hand-authored behaviour is suboptimal, and his practical objection is that it also stops scaling. Complex agents have sub-agents with prompts in many places, so a change in one spot breaks something somewhere else.

GEPA and Harness Evolution

Compute can be applied to all three parts, and the interesting ones are the two people usually leave alone.

• The model - post-training the policy with supervised fine-tuning and reinforcement learning on data for your distribution, which is what the frontier labs do at scale and what an increasing number of companies now do for themselves.
• The prompt - GEPA uses other LLMs to reflect on the agent’s failures and iterate, with an evolutionary algorithm driving the search. It spends compute in the form of tokens and sandboxes to find prompts a human would not, and there is a standard library for it.
• The harness - most teams treat it as fixed, and he sees no strong reason to. Adding tools built for the specific task is one route, and applying the same evolutionary techniques to the harness code is where the research is heading.

The Scarce Ingredient

Laid out that way, his central claim follows from what is already commoditised. Compute is accessible, and the optimisers, the fine-tuning libraries and the reinforcement learning libraries are well understood and widely available.

For an enterprise that reframes what is worth protecting. Your task is your recipe and your intellectual property, so the environments capturing your use cases are the asset rather than the model you rent.

Snowflake’s 103 Environments

His first case study is data engineering, which is a large use case for Snowflake and a different distribution from what a general coding agent was trained on. Bespoke curated the environments; Snowflake is training its own agent to be good at that work.

With 103 RL environments in place, evaluation becomes systematic rather than impressionistic. Snowflake’s CEO shared a comparison of two frontier models across that set, which circulated widely.

What that comparison yields is more than a leaderboard. Success rate per model, token usage and cost, behavioural differences such as one model doing considerably more verification before answering, and failure-mode analysis from reading the trajectories.

From there the same environments feed improvement rather than just measurement, whether through post-training an open model, evolving the prompts, or adding tools to the harness such as query runs that verify their own results.

Credit Karma and Compliance

The second case study is Intuit’s Credit Karma, where the product recommends a credit card from your usage and credit history and, more delicately, explains why.

It had already shipped before Bespoke was involved, and it was hallucinating. Card names written incorrectly, typos in interest rates, an offer of a 0 percent rate that was not real, alongside latency high enough to hurt the experience.

His framing of the stakes is what makes this different from a quality bug. A hallucinated rate is a compliance violation, which upsets the card issuers and runs into government regulation.

The cause fits his thesis. This use case was not in the training data for frontier models, so it is a new task distribution and needs data curated for it.

They post-trained an open-source model on curated data using supervised fine-tuning alongside GEPA. Hallucinations dropped substantially, and because the model is open and smaller, cost fell, throughput rose and latency came down, for a product serving more than 100 million users.

The Data Flywheel

His closing recommendation is a role rather than a tool. Become an RL environment engineer, on the grounds that most of the surrounding work is being automated and this part is not.

Where LLM work meant writing training data and letting GPUs fit the shape of the task, agent work means building the environments and letting training and evolution fit that shape. Prompt engineering was the manual version of the same job.

The loop is what makes it compound. Deploy, read the production trajectories, find where the agent fails, and curate new environments aimed at those failures.

The alternative is what most teams have now. A frontier agent deployed as-is is static, and its performance does not improve over time beyond whatever you adjust by hand.

Q & A Session

Four audience questions closed the session.

• How do you test rare but high-impact agent failures?

  Mahesh Sathiamoorthy: The common approach is to deploy and see what breaks, and the alternative is the older machine learning discipline of train, eval and test data before anything reaches an A/B test in production. Curate environments as close to your production reality as you can, drop the agent in, collect the trajectories, then use other models to cluster and analyse them into failure modes. He pointed out this is what a benchmark is really doing: an agent scoring 80 percent on a benchmark leaves 20 percent of failures you can study. There is art in it, and it can be made far more systematic than it usually is.

• When does soft benchmarking apply versus hard benchmarking?

  Mahesh Sathiamoorthy: He asked for the terms to be clarified rather than guessing, then took soft benchmarking to mean what you do before production and hard benchmarking to mean examining production failures. The goal is to move as much work as possible into the first so the second holds few surprises, which comes back to curating enough tasks resembling what you will actually see.

• Scaling to over 100 million users at lower serving cost, were trade-offs made on model size, context length or guardrails?

  Mahesh Sathiamoorthy: On size, they moved from a much larger proprietary model to a smaller open one, on the reasoning that a system recommending credit cards does not need all the intelligence in the world. With train and eval data in place it becomes an evaluation exercise: train several sizes, plot cost against return, and you may find a cheaper point delivering 80 to 90 percent of the larger model’s impact. On context, the previous setup needed a very long prompt listing every rule and compliance requirement, which drove latency up and, he suspects, hallucinations with it. Post-training moves that knowledge into the weights, so the long context is no longer needed, and since attention cost grows with the square of sequence length the latency reduction compounds. On guardrails, far fewer were needed once the model itself measured better on compliance.

• How does memory overhead scale when running thousands of environments simultaneously?

  Mahesh Sathiamoorthy: Most of these environments are virtual machines, so running thousands means spinning up thousands and the CPU cost becomes real. Around that you need infrastructure to snapshot and restore, because an agent failing halfway through is a state you want to recover rather than restart. He noted the sandbox companies emerging to serve exactly this, naming E2B, Daytona and Tensor Lake, and said Bespoke runs its own. His framing of the cost is that it is the trade you make to improve systematically before production rather than after it.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.