Redesigning End-to-End Testing in the Age of AI [Testμ 2026]

Every Box Automated

Sharma opens with a list of what he believes the industry has genuinely solved: test execution, regression, reporting, test data, test design, script generation, defect analysis and defect reporting. Automation and AI made all of it work.

He hedges his own coverage claim rather than leaving it absolute. Automation is in place in most teams, not every team, because he still knows a couple that do not automate extensively.

He states what the talk is not before he starts. He is not there to tell anyone that automation is the future, or that AI is what they need to use, treating both as settled by other sessions at the conference.

The distinction that carries the rest of the session is between local testing and end-to-end testing. Checking that something you built works on your machine is fairly simple, because interdependent systems, people availability, data ownership, environments and tooling do not bite at that level.

The host, a marketing manager at TestMu AI, which publishes this recap and ran the conference, delivers the same thesis in the intro: teams optimise the task and never touch the gaps between them, and become the invisible integration layer that no dashboard records.

Sharma is insistent about the word. End-to-end testing does not merely feel manual to him, it is painfully manual, and he says he has been speaking about it at conferences for years.

Day One Of An E2E Cycle

His scene-setter is the Teams or Slack thread that opens every cycle, where the whole team asks whether the environment is up, whether anyone is watching the logs, whether the right systems are connected, and whether the middleware is working.

His worked example is an SAP integration. Data was sent from the front end, and somebody has to confirm the SAP system received it. The harder case is the one where an interface reports that it exited the data while the downstream validation system says nothing arrived.

He stresses that all of this happens with everything nominally in place. There is an end-to-end release plan, logs are activated, every ticket is open in Jira, pipelines are set up correctly, and the technical implementation of automation and agents is done well.

The pain he describes is diagnostic rather than technical. With no answer available you go looking for the single point of contact for that system, chase several team members, and often find the fault sits in the upstream system feeding the data, not the system under test.

He generalises the claim widely, across project, product, team, sprint and scrum models and whatever technology is involved. No survey or dataset sits behind that generalisation, and he offers none.

The Mars Orbiter Arrow

The anchor case is NASA’s Mars Climate Orbiter, which Sharma says launched in December 1998 and was meant to enter Mars orbit rather than land, relaying climate data and acting as a communications relay for a larger follow-on mission.

He walks through why delivering a spacecraft is not point-and-shoot: communications, environmental conditions, sunlight, gravity, the engineering of the craft itself, a nine-month cruise, and repeated thrust decisions to hold the required velocity.

The mechanism he names is angular momentum desaturation. The spacecraft transmits thrust values, Earth receives them, and the next thrust is computed from what was received.

The failure in his telling is a unit mismatch across that boundary. The transmitting system worked in pound-force seconds while the receiving system expected newton-seconds, at 4.45 newton-seconds to the pound-force second, so a value of one was read as one and the error compounded until the orbit was lost.

His punchline is the whole talk in miniature. Both components were correct, neither sent a false signal, and both were telling the truth. What failed was the communication between two components, two systems, two phases.

Two details from this passage are left out of this recap rather than repeated. He gives three different failure dates within a few sentences, spanning 1995 and 1999, and a target altitude figure that does not match the published record for the mission.

Boxes Versus Arrows

Sharma maps the testing lifecycle as a chain of blocks: requirement, design, data, configuration, execution, failure analysis, defect, reporting. For each block he names the automation that already exists, and for each connection he names the person still doing the work.

His verdict on the left two columns is that they work. That, he says, is exactly what he means when he says the industry automated everything, and why the pain is confusing to people who look only at their tooling.

His verdict on the right column is that it has barely moved in ten or fifteen years. He closes the section on a brick wall: you can count the bricks and check the quality of each one, and the cement holding them together still decides whether the wall stands.

Engineers As APIs

The claim is not that people do the work inside each phase. It is that people carry the payload between phases, translate it, and decide what happens next, which is the job description of an integration layer.

At the data phase the payload is tribal memory. Somebody who has run this before remembers that a particular dataset caused a problem two cycles ago and says so out loud, and that is where the knowledge lives.

At configuration and execution it is infrastructure hand-holding. Where to point the database, what happens when the team moves from one database to another, what to do about Docker, and where to run tonight given that staging has been down for three days.

At failure analysis the human is the classifier. Somebody checks each failing step manually to decide whether the cause is the environment, the system, the automation script or a genuine defect in code, and no agent in the chain has that answer.

The reporting example is the one worth keeping. A run marked complete does not mean passed, so a person has to explain that it completed, partially passed, and is being signed off conditionally. That sentence is the arrow, and no dashboard emits it.

The Beckham Rule

Sharma runs an extended football analogy, tracking David Beckham from Manchester United to Real Madrid and then to LA Galaxy in Major League Soccer.

The mechanism he cares about is the rule change. The MLS salary structure could not accommodate a European superstar, so the league created the Designated Player Rule, which became known as the Beckham rule, and which he credits for Messi playing in MLS today.

The mapping is direct. AI is the signing: powerful, capable, and holding everything needed to succeed.

The inversion he sees in practice is that teams buy the player and keep the league. Copilot access, agents, Claude subscriptions and every new model get handed to testers while the testing process, the systems, the structure and the architecture stay exactly as they were.

His counterfactual is that had MLS insisted Beckham fit the existing budget, none of it would have happened. His closing claim on the analogy is an absolute one, that every team today is AI enabled and none of them is producing what that one signing produced for the league, which is his assessment rather than a measured result.

The Demonstration Effect

Sharma borrows the demonstration effect from economics, defining it as adopting something not because you established the need but because somebody demonstrated it. He cites no author, paper or source for the term.

His illustration is domestic. A family has a good house, a cycle and a routine that works, with no problem at all, until a neighbouring couple arrives with a bigger house, a car and regular dinners out.

Nothing about the first family’s needs changed. The only thing that moved was their reference point, and now they want to be like the neighbours.

Organisations copying from a conference talk, a LinkedIn post or a summit run the same three steps in his telling: they see, they compare, they desire.

The agent arms race falls out of that. Somebody has five agents, so the answer is seven, and whether the seven are needed is not part of the decision.

His only supporting evidence is anecdotal, and he presents it as such. Almost everybody he meets has access to GitHub Copilot, and he does not know how many of them use it anywhere near its potential. Capability, in his phrasing, spreads faster than the need for it.

Occam’s Razor, Read Properly

Sharma pushes back on the popular reading of Occam’s razor. It does not instruct you to cut a lot of things, it removes the unwanted and unnecessary while leaving genuinely required complexity intact.

His worked example is an aircraft that needs a thousand components. The razor does not say build it with fifty, it says keep the thousand because each has a reason to exist, and do not add the thousand and first.

Applied to testing, the unnecessary additions are the handoffs scattered across every phase of the software testing lifecycle, which is where he thinks the simplification belongs.

His mantra for the current moment follows from that, and it runs against most conference advice. Enough has been automated, enough agents exist, and the work now is removal.

What he proposes to remove is specific: the manual handoffs, rather than any of the automation already in place. His reading of the razor is his own interpretation and not a citation, and he does not attribute it to a source.

The Handoff Tax

Sharma coins the phrase on air and says plainly that no such term exists. A tax is what you pay in order to do something, and this one is levied on every crossing between phases.

He names five currencies it is paid in: time, context, energy, ownership and accuracy. Ownership is the one most teams recognise last, because a handoff is also the moment nobody is responsible.

His decision rule for any new automation is a two-part test. Even where automating is reasonable, ask what additional handoff the automation introduces, and ask how much time and value would be saved by not automating it at all.

His measurement critique is the reason the tax stays invisible. Execution time is measured and reported accurately, delays between phases are not measured, and coordination is not measured, and he says he knows of no company, project or report that measures it.

The arithmetic he uses to illustrate it deserves a caveat. Twelve test cases at one case per three minutes gives 36 minutes of execution, which is the whole report; add roughly two minutes of coordination per case plus waiting on people and the elapsed figure balloons. He offers 65 days as the endpoint and hedges it aloud, so treat the pairing as a rhetorical contrast rather than a measurement from his team.

The Formula One pit stop closes the section. Picture the mechanic on the right wheel finishing, raising a Jira ticket, and telling the mechanic on the left wheel to pick it up. Pit crews engineered their flow rather than only their individual tasks, which is the distinction he wants applied to testing.

Main Tester United

The framework Sharma’s team built and uses is called Main Tester United, a Manchester United pun where the tester is the main part and automation and engineering are united behind them. He describes it as the framework they are using right now.

He is candid that the boxes are unremarkable. What his team automated is much the same as anyone else would automate, and the part that matters is what they kept, which is the context.

Context travels left to right through every agent and every phase. The behaviour he uses to make it concrete: if the requirement says a test case must be added, and the next agent holds the same context and can see that case already exists, it runs the case instead of creating a duplicate.

Traceability is revived deliberately. He reaches back to the requirement traceability matrix of the 1990s and applies that concept inside the AI chain, connecting requirements through to the logs produced after execution.

Defects are filed with logs, screenshots and videos, and reports carry contextual information about failures, blockers and dependencies rather than a count of seven passed and two failed.

One thing this section cannot give you is evidence. Sharma pre-framed a demo, asking the audience to watch the handoffs rather than the agents, and no demo follows anywhere in the recording. There is no architecture, no stack, no build history, no team size, no adoption figure and no before-and-after number attached to the framework.

The Bullwhip Effect

Sharma stress-tests his own framework rather than selling it. If context flows left to right, everything downstream depends on the first context being right, and the first handoff is where human error enters.

He credits a speaker he heard at another conference, whose name is not recoverable from the recording, with the line that a defect is not born in your code but in your missing conversations and your requirement gathering.

His corollary is an absolute claim worth flagging. Coding defects barely exist any more, he argues, because coding is no longer done by humans, so the radio button that does not click and the checkbox that does not check are behind us. Many practitioners will contest that, and he offers no data for it.

The bullwhip case is offered as a real COVID-era story with no company, country or source named. A customer asks for 7 packets of toilet paper instead of his usual 1, the retailer orders 70 rather than 20, the distributor orders 700, the manufacturer produces 7,000, and the customer no longer needs any of them.

The mechanism is named from the object. A small movement at the handle becomes progressively larger along the whip until it is enormous at the tip.

Applied to an agent chain it is the risk that context flow creates. If the builder hears X when the customer needed A, X is elaborated, X is built, X is executed, the results are analysed against X, and a defect is found somewhere inside X. One wrong interpretation enters and everybody downstream is wrong together.

The Autonomy Bliss Point

Sharma names the contradiction in his own talk before anyone else can. He argued humans should not be standing in the middle, then argued that full autonomy breaks on a single bad input, so neither extreme survives.

He borrows the bliss point from food product development and is explicit that his version is improvised, saying it is not a theory and that he made the analogy himself. The sweet spot differs per team the way it differs per product.

His rule is that the level should be neither maximum AI nor minimum human, but a designed, decided and dedicated level of autonomy for your particular situation: enough that context flows, and enough human judgment that decisions stay intentionally human.

The three questions he wants asked of any AI testing product replace the usual two. Instead of how much AI is used and which model it runs on: who owns the context, who owns the decisions, and who owns the consequences when the machine decides wrongly.

Handoffs split into two kinds on that basis. Intentional handoffs are kept, and accidental ones are engineered out, and every accidental example he gives is copy-paste work: lifting values out of logs, pasting an API payload into a Postman collection, hand-translating XML into JSON or JSON into YAML.

He clarifies his own session title at the end, because it reads more aggressively than he means it. The last manual handoff is not about removing the human, and his position is that intentional human handoffs should stay.

His close reframes the lifecycle. It is no longer blocks, it is blocks connected by arrows; the next round of improvement sits between two test cases, two systems, two decisions and two engineers; and the useful question is no longer what to automate next but how to bring continuity to everything already automated.

Q & A Session

Five audience questions and one moderator follow-up closed the session.

• Which AI testing practices do you recommend for enterprise teams, such as DevOps, guardrails and red teaming?

  Sachin Sharma: This is apart from my session, but the practice needing more attention at enterprise scale is tokenisation. Capability is not the constraint, cost is. There are two realities to hold at once: the one the tool promises and the one the budget allows. I cannot give you a specific recommendation because I do not know your company, your domain or your project. Budget is the only constant, and everything else is already sorted. None of the three practices named in the question were addressed.

• What is one mistake QA engineers make when they start using AI in testing?

  Sachin Sharma: Outsourcing the thinking. The engineering judgment that should stay human is the first thing we hand over, and we skip the question of why the problem exists and go straight to how to solve it. Add two more team members, twelve agents and premium Copilot access, before anyone has asked whether the problem is real. The model’s own confidence is part of the trap: a system that answers anything with certainty invites being trusted with everything.

• Moderator follow-up: any other perspective on the first mistake people make?

  Sachin Sharma: This will sound funny, but the first mistake is still typing please and hi to AI. It is unnecessary, and reports I have read suggest even a greeting consumes a bucket of water. No report, study, figure or publication was named, so that last claim is unsourced and should be read as one.

• Which part of end-to-end testing is hardest to fully automate, even with AI in the loop?

  Sachin Sharma: In our case none of it is fully automated. The hardest part is the one that sounds easiest, generating test cases, because the artifact is not the script but the reasoning: what you want to test, the data behind it, and the thought process behind why the other cases are not being tested. Guardrails and execution-style instructions still are not enough. AI will never produce a test case that no experienced tester needs to review, and we should stop trying, because this is not a problem to solve and the human is required. Every tool I have seen keeps a human in the loop at exactly that step, which is an unquantified observation rather than a measurement.

• Can AI understand business-critical user journeys better than traditional automation?

  Sachin Sharma: Yes. Although I do not know. I have stopped doubting AI capability: within three or four months I moved from assuming it could not do something to believing anything can be done for the testing industry, and I am saying that even when I do not know the answer. That is a non-answer, and no comparison against traditional automation on business-critical journeys was made, which is what the question asked.

• Can it be integrated with Azure DevOps, as we do not use Jira?

  Sachin Sharma: Yes. Jira, Confluence and databases are just the platforms we happen to use; the agents can integrate with any tool, DevOps or Azure DevOps included. Two people from my community have done comparable Azure DevOps tooling work, and one of them presented here yesterday. No integration detail, connector list or caveat was given, and those two names are not reproduced here because he twice said on air that he did not have their permission to name them.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.