The Full Agentic QA Loop for Web Applications [Testμ 2026]

The False Sense of Proof

AI has been writing code in seconds for about two years, and Siddhant Sinha is not arguing with that. Claude Code, Codex, Gemini and Copilot all get used to build harnesses, write unit tests and wire CI.

His target is narrower and sharper: the same agentic loop that generates the application then asks that same AI to write and prove the test for it.

The tests that come out of that arrangement are brittle, slow, and never actually proven in a real browser. He says the team spent seven to eight months trying to close the loop end to end, which is where Kane CLI came from.

The PRD-to-Test Gap

The second problem is organisational and will be familiar anywhere with more than one job title.

Product managers, developers, QAs, SDETs and leads all work from a single artifact, the PRD written for a use case. The developer builds the feature from it while the QA writes test cases from it, and the translation from business use case to executable test is done entirely by hand.

What is lost in that translation is the link. Nothing connects the business use case to the scenarios and acceptance criteria meant to cover it.

The symptoms he names are drift, stale tests, and coverage claims that are guesses. Saying a feature is 70% covered when nobody can show the mapping is his illustration of the problem rather than a measured figure from anywhere.

The Loop at a Glance

The workshop walks one path in order, and it is worth seeing the shape before the detail. Each stage produced something concrete against the demo PRD.

TestMu AI sits behind the CLI as the test management system, so authoring creates the test case centrally and even a local execution creates a test run, while the evidence itself is emitted locally.

Ingesting the PRD

The demo application is a locally hosted food-ordering site, and its PRD is a real document rather than a one-liner: the use cases built, the kinds of tests wanted, the applicable rules, how the order number is generated, and which network calls occur.

Ingesting it produces use cases, not tests. That ordering is the design decision the rest depends on.

Four came out: placing a food order for delivery, marked high risk; signing in before ordering; managing the cart, covering quantities, removals and totals; and browsing the menu to add items. The last two arrived marked medium.

Each carries a summary the design phase later consumes, and reclassifying one takes plain English. He typed that use case three should be high risk, and the agent re-checked the request and promoted it.

His framing of what the extraction replaces is worth quoting back at anyone who thinks this is autocomplete: it is the work a QA lead or product person with ten to fifteen years of experience would do with a single PRD.

The Review Gate

Asked to design, the CLI refuses and asks for approval first, because the use case has not been reviewed.

He knows what was created, so he approves inline and design proceeds. The explicit path is a review command followed by approving a specific use case, which records that a human was satisfied with it.

Approval and design are deliberately decoupled, and his reasoning is practical rather than architectural. No QA lead, PM or manager designs four extracted use cases in one sitting, so sessions hold the state and you resume where you stopped.

Mid-design the CLI shows a session meter: phase two of four, seventeen items in the session, and the credits consumed so far for designing this one use case. Cost is visible while the work happens rather than after it.

Thirteen Acceptance Criteria

Designing against use case one produces 13 acceptance criteria, and reading them aloud is what makes the abstraction concrete.

Checkout shows a full name field. Checkout shows a delivery address. Card and cash are both offered. Card is the default when checkout opens. Place Order stays disabled while the full name is empty.

The success path gets its own: a confirmation message, an order ID on placing the order, the total paid shown on the confirmation screen, and a back-to-menu action returning the user to the main screen.

One criterion is a useful reminder that this is not a UI-only tool. Kane CLI inspects network, console and dev tools and can assert against them, so one criterion here is that no network calls fire, because the demo application is static.

Design stops there on purpose. Acceptance criteria state what needs to be tested; scenarios are a separate step.

The Agent’s Two Questions

Before building scenarios the agent stops and asks, and he uses the moment to make a point: this is not magic, and a better PRD would have produced fewer questions.

The first concerns the rule that Place Order is enabled only when name and address are non-empty. Should whitespace-only entries count as empty, or as filled? He takes the recommended option, literal non-empty.

The second is the scenario budget, where the agent proposes four: two happy paths, one negative and one boundary case.

He answers no limit rather than capping it, and the reason is the most instructive detail in the workshop. Capping the count makes Kane CLI record coverage gaps around the scenarios it was not permitted to write, so an artificial limit shows up later as a hole in the coverage report rather than disappearing.

Wiring Scenarios Into Tests

Wiring is the step he calls the genuinely hard part, and it visibly takes time on stage: deciding which scenario covers which acceptance criteria.

The output is four tests, each a scenario paired with a specific set of criteria it verifies. The checkout that succeeds on the default payment method and clears the cart verifies eleven. The negative test, where checkout stays blocked until both required fields are filled, verifies six. The test that switches payment method verifies twelve, as does the boundary test accepting one-character fields.

He is upfront that this is a small example and the value shows at scale, when several PRDs arrive and use cases overlap. The linkage Kane CLI maintains is exactly the PRD-to-test mapping teams otherwise keep in a spreadsheet or in someone’s head.

The Graph and the Test File

After design, listing the context shows scenarios, acceptance criteria and tests alongside the source and use cases that were there before.

The graph view makes the relationship visible: the ingested PRD linked to all four use cases, each use case to its scenarios, each scenario to the test created for it, and each test to the criteria it verifies, all clickable.

Lineage is empty at this stage. It appears once you update the PRD and re-ingest it through the maintain path, which evolves a specific document and records its history, which is the answer to the drift problem the session opened with.

Opening a generated test in the editor shows plain English, no locators, numbered steps. One step declares which scenario and which acceptance criteria that test verifies.

Because that file format is simply what the CLI understands, the whole assurance path is optional. You can hand-write a test file describing what you want tested in plain English and run it.

Discovery, Then Replay

He picks an authored test and runs it with no extra configuration, splitting the screen so the browser and the terminal are both visible.

The run decomposes into seven intents. The first handles sign-in and details, the second adds items to the cart and verifies them, and one negative check runs live: going to the cart without adding anything and confirming it is empty.

Underneath, the agent is reading the screen, the tree structure, the network logs and the console logs, and resolving each intent into concrete actions itself.

The distinction he draws here is the one that answers the usual objection about agentic tests being slow. The first pass is discovery, where the agent works out how to run the plain-English test, and it is slower by design. Everything after that is replay of what it already learned, and the re-run is visibly quicker.

Evidence and Coverage

A successful run saves artifacts, on the principle that without evidence nobody can see what was executed.

Test Manager shows what was tested and offers a replay view, which is not a code run but a view of what the agent did, verifying and going back to check, alongside emitted deterministic code and a version history tracking how a test changes over time.

The evidence pack itself was served from his own machine during the session rather than from the cloud, and a hosted viewer lets anyone you share a pack with drop it in and see the run. The view separates the authoring steps from the execution steps.

Coverage is expressed against acceptance criteria rather than test counts. One criterion is linked to all four tests, another to three, and the report shows which criteria a given run actually verified.

The headline from the single executed test was 12 of 15 acceptance criteria verified, offered as the answer to what the coverage of this product is, where passing criteria mean working tests and failing ones pull the number down.

One number does not reconcile on air, and it is worth flagging rather than smoothing over: design produced 13 acceptance criteria for use case one, while the coverage screen reports a denominator of 15. The session never explains the difference, so treat the 12 of 15 as what the tool displayed rather than as arithmetic you can follow from the demo.

Q & A Session

The workshop ran out of time before the agent-integration section, which was promised rather than demonstrated. Three audience questions were taken instead.

• How does agentic review detect that a test has drifted out of sync with the application?

  Siddhant Sinha: Kane CLI treats the PRD as the source of truth, so a test built from it that starts failing means either the PRD is wrong or the implementation is. Drift specifically means the feature changed, which means the PRD needs updating and re-ingesting; once it is, the drifted test can be identified and fixed. Skip that and the test simply keeps failing, and the failure shows up in the coverage number until the changed feature is handed back to the tool.

• When the agent updates tests to match evolved behaviour, what stops it silently masking a real bug?

  Siddhant Sinha: Pass and fail are defined at the acceptance-criteria level, and the guardrail is that criteria approved by a user cannot be modified by the authoring agent or the execution agent. The agent can still take its own actions and can still make mistakes, but it can never rewrite the criteria, so anything outside the intended flow starts to fail. He says that immutability is what his own testing team relies on. Worth noting this is a design assertion made in the Q&A rather than something demonstrated on stage.

• Who is responsible when an autonomous agent causes a security incident?

  Siddhant Sinha: For Kane CLI specifically, he says the tool has no access to anything beyond the PRD or context you give it. Speaking generally, he puts responsibility on whoever manages the agent, on the grounds that humans set the guardrails and decide what access an agent gets, so whoever holds that decision owns the incidents that follow from it.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.