Scaling Trust, Not Automation: Rethinking Quality for the AI Era [Testμ 2026]

Confidence Is Not Correctness

The hiring story behind the talk is a small case of confident wrongness. In late 2018, living in Ireland, at the fifth and final interview of the Amazon loop, the hiring manager told him his German-speaking skills would be really helpful. He is Argentine, spoke no German, and said so on the spot.

The wrong bet paid off anyway, because the Alexa Spanish launch was on the roadmap and he is a native Spanish speaker. He says he was the first person able to hear Alexa speak Spanish end to end on a device, and went on to spend eight years on Alexa quality. The right outcome for entirely the wrong reason.

To make it concrete he played a 2016 clip in the session, in which a small child’s request was misrecognised and the device began reading out a list of explicit results until an adult said “Alexa, stop”.

That incident kicked off an internal effort producing what the team called earcons. Most natural language classifiers return a confidence score between zero and one, and the team added a threshold: below it, the device deflects with a tone and stays quiet rather than acting.

He flags the subtle cost of that fix. The system stopped failing loudly and started failing politely. The error did not go away, it got quieter.

Even with the safeguard, he recounts a 2021 incident in which a child asking for a challenge was told to put a penny into a power outlet and touch it. The child told her mother and was safe. He recalls it second-hand and without citation, and his point is not an attack on a product he worked on and says he loves: if this can happen on one of the oldest and most heavily tested voice assistants on the market, it can happen anywhere a probabilistic system sits in front of users.

Lessons From Bad Early LLMs

He started experimenting with LLMs in late 2020, in what he calls GPT-2 time, and in late 2021 demoed that work to leaders at an internal Amazon summit: test case generation plus early experiments using language models for classification and assertions.

The chronology is the interesting part. ChatGPT arrived in November 2022 and the LLM-as-a-judge term was popularised in June 2023 by the MT-Bench paper, which means his December 2022 automation framework already had synthetic customer simulators and a first set of LLM-powered judges before the vocabulary existed.

He frames the timing as luck rather than foresight. The models were bad, really bad, so he never had the option to trust them and had to question every result.

That habit produced the question he could not put down: how do we know we can trust these systems, particularly when we are using them to evaluate the quality of another system in order to decide whether to ship.

Defining Scale and Trust

Calling himself a lexicon nerd, he defines scale in the computer science sense as a system’s ability to handle growing workloads without losing performance, and notes the word has two roots: Old Norse skal, meaning bowl, and Latin scala, meaning stairs. He leaves the audience to decide which one describes their pipeline.

Trust he defines as a firm belief in the reliability of someone or something, where belief unpacks into faith or confidence, and reliability is the degree to which something can be depended on to be accurate.

Combine the two and scaling trust becomes a system’s ability to grow while sustaining a reliable belief in its accuracy and dependability as demand increases, without that belief degrading under the weight of the growth.

The takeaway he wants held is that trust has to be earned, has to be measurable, and has to compound. If reliability falls, trust will not scale, which makes trust a dynamic property of the system from day one rather than a label applied after the fact.

Rise of the Applied Scientist

The applied scientist’s role, practices and jargon are creeping into testing. He now uses precision, recall, F1 and specificity in conversations where five years ago he would have said pass and fail, and sees test catalogs becoming data sets and test suites becoming test sets, a rename he likes.

He pushes back very hard on the claim that traditional testing no longer serves a purpose because models do not have binary outputs. He says he hears it from smart engineers, including in a strong argument with a junior engineer two weeks before the talk.

His counter is that the application under test still has a job to do and the user does not care how it works. Book a flight and the reservation has to be real. A purchase has to charge the right amount and deliver the right product on time. An alarm set through a voice assistant has to ring, and a calendar invite has to be reflected on the device.

Functional and quality-attribute testing therefore matter as much today as five years ago, and the goal of testing has not changed.

What changed is that stochastic components now sit inside the system and need their own verification methods. He calls that far more precise than “testing is dead”, and unlike that phrase, it leads somewhere.

The Volume Trap

When he joined Alexa in 2018 his team owned testing for international expansions, and his manager wanted a suite that ran continuously across every supported locale. No tooling existed on the market and almost no literature described how to test a system like Alexa, so they built their own. He says the literature still barely exists.

The question that worried him then was not how to generate tests but how to review the results, and it maps directly onto now. With generative AI everyone produces thousands of test cases before launch, which sounds excellent until 5% fail and you are staring at 3,000, 4,000 or 5,000 failures with no idea which are real, which are hallucinated, or how to prioritise them. Those numbers are his thought experiment rather than measurements from a system he ran.

He runs the denial list explicitly. More tests do not mean better understanding, more coverage, fewer bugs, a more stable product or happier customers.

Volume has always been a vanity metric, because it measures effort rather than value, and volume without interpretability is noise with a confidence score attached.

The old principle that exhaustive testing is impossible still holds. The only change is that teams can now fail at exhaustive testing much faster and much more expensively, because every one of those tests carries a token cost.

Testing Versus Evals

He draws the line as a difference in question. Testing asks did it pass; evals ask how good is it and how do we know. Traditional QA gave you green or red, evals give you a distribution, and he says most teams are not ready for that conversation.

Evals do not replace testing. They are different tools for different jobs, and combining them carelessly is where he sees teams get into trouble.

The healthy connections he endorses run both ways: test executions produce data that is good for evals, and eval results tell you what to test next and when. What he rejects is the idea that either one subsumes the other.

His worked example: when a platform says an item has been added to the shopping list, validating that the item is actually on the list and displays correctly is testing.

Judging whether the assistant’s response is culturally appropriate for the locale, or whether the accent and pronunciation are right, is an evaluation. Both matter, because they answer different questions.

Probabilistic vs Binary Gates

The core tension is structural. Pipelines, release gates and stakeholders all want a yes or a no, while the systems being built produce probabilistic outputs and are handed to consumers who need determinism. Set an acceptable threshold and call it a day, and you have still reduced a distribution to a binary somewhere in the pipeline. That reduction is where trust starts breaking.

Percentages are hard to reason about, so he reframes them as questions. What if the 1% that fails is the most critical path in your system? What if a scenario fails 0.1% of the time on a feature used millions of times a day? That is not an edge case, it is a headline waiting to happen, like the clip he opened with.

A traditional failing test throws an exception and stops the pipeline, and with decent traceability you can usually find what went wrong fairly quickly. An LLM does not fail that way. There is no exception, no red light, just a sentence that sounds and looks amazing and is completely wrong.

Then comes what he calls an uncomfortable story. He recently demoed a proof of concept for a new testing approach, validated the results with QA engineers, re-ran the same tests manually on existing infrastructure, and confirmed the issues it found were real and useful. The solution uses agents in several places.

He is not confident it is reliable enough for production or to replace the old system. Some stakeholders heard that the proof of concept found real bugs and translated that into productionising it immediately, and he had to be the person saying no. Finding real bugs in a demo is not the same as being trustworthy in production.

The Compounding Math

The arithmetic is deliberately basic. Start with one component or agent that is 95% accurate, whatever accuracy means for your system, which is a number most teams would be very happy with.

Add a second component that is also 95% accurate and ask what the pair scores. The instinctive answer is 95%. Assuming the components fail independently, joint accuracy is the product of the individual accuracies, and that independence assumption is doing real work here.

The obvious escape is shipping fewer, bigger agents that do everything, and he rules it out at the current state of the art. Even as models get smarter and context windows get bigger, research keeps showing that agents trying to do too much degrade, a phenomenon called context rot. He says there is good empirical work behind it and invites the audience to look it up, without naming a paper.

In his experience, observing and debugging a single agent that does everything is also much harder than debugging a pipeline of focused components.

The conclusion he draws is the pivot of the talk. Trust cannot be hand-waved at system level. It has to be earned at the seam between every component.

Evaluation-Driven Design

His first prescription sounds obvious and is not: measure from the start. Engineers, himself included, tend to be solution-oriented before they are problem-oriented, and it is very easy to build a quick prototype.

He cites data going around that 95% of AI startups fail and that the 5% which succeed solve real problems for real customers. He names no source for it, hedging it as data going around, and takes the general lesson: know the problem before you build the system, and know what success looks like before claiming you have achieved it.

For stochastic systems that means data sets and ground truth, even a small scrappy version, because without a baseline you cannot tell whether the system is improving, degrading or just changing. That baseline cannot be retrofitted.

He is a TDD practitioner who still writes tests first even though he estimates AI now writes 95% of his code, a personal estimate about his own workflow. The analogue he points to is evaluation-driven design, a term he credits to others rather than himself.

The loop runs like this. Before building the agent or component, write examples of what good looks like: inputs, expected outputs, expected properties of outputs. Treat that eval as a test, build a harness, and run it so it fails, which prevents trivial always-passing evals. Build the smallest thing that turns it green, often a very simple or even hardcoded prompt, which proves the wiring and infrastructure calls are right. Then refactor, which in the agent era means a cheaper model, a leaner prompt and fewer tools.

The crucial difference from TDD is that green is probabilistic. You are not asserting equality, you are asserting that a distribution over outcomes meets the threshold for that agent, so every refactor has to account for variance as well as correctness. The eval becomes the safety net and gets committed alongside the component.

The Double-Loop Model

Once there is more than one component, he adds a second loop around the first, the double-loop TDD shape that anyone who lived through microservices will recognise.

The inner loop is component level: fast, focused on each component’s contract.

The outer loop is system level: slower, more expensive, run less often and only when there is a reason, explicitly not around the clock. Its job is to catch the cases where every component passes its own evals and the system still produces garbage, costs too much, or loops forever.

The two loops talk to each other. When the outer loop catches something, diagnose which seam broke and write a new inner-loop test that would have caught it, exactly the way production bugs become unit tests in a mature codebase. Over time the inner loop gets denser and the outer loop catches fewer surprises, which is what maturity looks like for these systems.

He is candid about what he cannot answer: how to version evals when the prompts underneath them change, how to keep the outer loop affordable enough to run on every commit, and how to write evals for emergent behaviours where a swarm of agents does something none of the individual agents were evaluated for. He floats the idea of emergent bugs, which do not exist in isolation but which a swarm produces, and asks anyone with ideas to reach out on LinkedIn.

Contract Testing for Agents

The industry has been somewhere like this before, and he finds himself reaching back to what is now classic literature: Agile Testing by Lisa Crispin and Janet Gregory, and Continuous Delivery by Jez Humble and David Farley, because the shape of the problem is familiar.

When microservices took off the test pyramid got inverted and it hurt. Teams discovered full system integration tests were slow, flaky and expensive, and the bugs they caught were often the wrong ones. The fix was contract testing: pin the interfaces between two services and test each side against the contract independently.

For multi-agent systems he maps the contract onto the handoff between components, meaning the inputs, the agent specs and the output promises. Pipeline-level evals, he warns, are the flaky end-to-end tests of this era.

The second lesson is that observability beat testing in production. Because you could not pre-test every path through a distributed system, the industry moved budget from more tests to tracing, structured logs and correlation IDs. Multi-agent systems need the same discipline, with every prompt, tool call and intermediate output traced end to end under a shared request ID, because when something goes wrong three agents deep, reading the transcript is your debugger.

The third is that schema versioning was non-negotiable, learned the hard way when a breaking change to one service’s response format took down consumers nobody remembered existed. The agent version of that is versioned prompts, structured outputs with schemas, and tracking which downstream components depend on which upstream behaviours, because changing the system prompt changes the contract for every tool that prompt touches.

The meta lesson is where he thinks agent QA is headed: the centre of gravity moves from testing the whole thing to contracts at the seams, observing the flow and isolating the iteration.

The QA Scientist Role

The role-shaped consequence of all this is a job sitting between the QA engineer and the data scientist, which he has been loosely calling the QA scientist. He says he does not fully know what it looks like yet, but he knows what it needs to do.

It needs to reason about statistical significance, think about the contracts between components, and think in distributions over outcomes. Comfortable saying this is 87% reliable, and equally comfortable saying why that is definitely not good enough to ship.

He sees the gap forming from both sides: QA engineers teaching themselves statistics because the job demands it, and data scientists building strong models without a strong instinct for how to test them in production.

He describes sitting in a meeting where a QA department and ML engineers talked past each other for an hour because their working definitions of quality were different, and predicts some listeners will either grow into this role or hire for it for the first time within the next couple of months.

He closes by returning to the opening story, where someone looked at his surname, made a confident inference, was wrong and got lucky. The question is not whether you should trust that component, but how you are going to prove that you should.

The talk was pre-recorded, so it took no live questions. Walter Zimerman pointed viewers to a QR code and to LinkedIn instead, and the host confirmed afterwards that many audience questions had come in and would be picked up through TestMu AI’s channels.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.