Hero Background

Next-Gen App & Browser Testing Cloud

Trusted by 2 Mn+ QAs & Devs to accelerate their release cycles

Next-Gen App & Browser Testing Cloud
Testμ

Surviving the AI Code Tsunami [Testμ 2026]

Mathias Strziga on why a red test result is a classification problem rather than a bug, and where the human gate sits in an agentic QA pipeline.

Published on:

A test goes red. Before anyone can call it a bug, somebody has to work out whether the product is wrong, the spec is wrong, the test is wrong, the infrastructure fell over, the harness fell over, or nobody knows.

At Testμ Conf 2026, Mathias Strziga, Director of Quality Assurance at Appsfactory Group, built his whole architecture around that last category. Everything else gets handled inside the loop.

Youtube thumbnail

If you couldn’t catch all the sessions live, you can access the recordings at your convenience by visiting the TestMu AI YouTube Channel.

TL;DR

The AI DLC is Appsfactory’s internal development lifecycle for agent-written code, built from a set of skills, agents and workflows that Mathias Strziga counts at roughly 82, nine and more than six. It exists because cheap code makes specification and verification expensive, so effort moves to the two ends of the lifecycle.

  • Is a red test result a bug? - No, and this is the sharpest claim in the talk. Mathias Strziga treats a red result as a classification problem, because you do not yet know whether the use case or the circumstances were right. Every report is sorted before anyone repairs anything.
  • What are the categories a failing result gets sorted into? - Product, behaviour, specification, test issue, infrastructure failure, harness failure, or unknown. Mathias Strziga handles everything except unknown inside the loop by reevaluating, rewriting and retesting, with tests or bugs reported at the end.
  • Where is the human gate in an agentic QA pipeline? - At the unknown classification. Mathias Strziga says the unknown stops the chain and needs a pilot in the seat rather than a co-pilot, and that same human then decides whether the episode becomes a durable lesson or is discarded.
  • Why did specification get more expensive when code got cheap? - Because agents now write the code, so the effort moves to telling them what to build. Mathias Strziga says teams need to supply more specification, more context and more intent than ever before, and concedes this runs directly against the agile principle of discovering as you go.
  • What are bursts rather than sprints? - Mathias Strziga’s replacement for the two-week sprint: specify heavily up front, let agents build in three-to-four-day bursts, then verify at the end. He describes what emerges as an increment still, comparing it to Scrum on steroids.
  • Why run QA agents on a separate path from build agents? - So QA reaches its conclusions independently. Developers ask how to build it while testers ask how to challenge it, whether it is testable, and when it is good, and the two sides exchange information only after both have concluded, ideally agreeing.
  • What does a validation contract have to specify? - What is being validated against, what counts as an oracle, what is observed, and which actions are allowed. Mathias Strziga’s point is that validation runs against intent rather than strictly against code, in a model he compares to contract-driven testing.
  • Why can’t you keep all specs and intent in Git? - Because repositories are optimised for preserved results, so by design Git looks backwards. Holding every spec, intent and definition of done there would mean streaming the repo for changes and throwing the agents into deliberation over whether each change is relevant.
  • Does every step between agents need a stop condition? - Yes, and Mathias Strziga calls it very important. Each step carries scope, evidence and requirements plus stop conditions, so that a surfaced issue can be confirmed as a real issue rather than a one-time incident or a test system mid-deployment.
  • What does he say the future challenge is? - Convergence rather than speed. He points to many developers in a large corporation working on the same or neighbouring systems, plus third-party dependencies shipping on their own cadence, as the coordination problem that agent throughput does not solve.
  • Does the talk say QA jobs disappear? - No, it argues the opposite. Mathias Strziga says low-discretion work that used to be junior or training work is easiest to automate, while roles built on learning, decision-making and human judgement are still being created, closing on QA being promoted to the pilot seat.
  • Are there numbers behind any of this? - No customer numbers, no before-and-after metrics, no defect-escape rates and no cycle-time data appear anywhere in the session. The only figures are Appsfactory’s own self-reported framework and company counts, and Mathias Strziga hedges those as a snapshot from when he built the deck.

Effort Moves To The Edges

He opens on a slide of what he calls the legacy software development lifecycle, where the effort is always highest in the middle, during architecture and design, writing code, and the testing that follows.

Comma

The delivery shape that follows is bursts rather than sprints. Specify heavily at the start, let the agents write it in three-to-four-day bursts, then verify at the end, and it is that verification tail he identifies as the problem the talk is about.

He attaches a velocity figure to the burst model. The sentence is damaged in the recording, carries no baseline, sample or measurement method, and is the only performance number in the talk, so it is not reproduced here.

What emerges at the end of a burst is still an increment in his framing, which he likens to Scrum on steroids rather than a break from it.

He issues his own expiry date on the material early: he is fairly confident the talk will look substantially different by Christmas, because the field moves quickly. Every framework number below should be read as dated to the deck rather than to now.

The Framework In Numbers

The framework, which he consistently marks as his organisation’s own rather than an industry standard, is supported by roughly 82 skills at the point he created the presentation and probably a few more by the time he presented it, nine independent agents, and more than six workflows.

Those hedges matter. The published chapter list flattens them into a clean 82 skills, nine agents and six workflows, which drops both his snapshot caveat and the more-than on the workflow count.

He says the framework splits into three phases, then lists four things: specification, implementation, testing, and maintenance and operation. The description resolves the mismatch by deleting testing from the list, which is a strange thing to drop from a QA talk. He never reconciles it himself.

The nine agents are counted but never named or enumerated. The supporting visual is a word cloud, so no reader can reconstruct the inventory from the recording.

The design goal he states for every skill and agent is feedback: each one feeds information back into the wider system, so the intent is accumulated experience rather than task execution alone.

On his own organisation he gives about 450 employees, originally a mobile app specialist and now full-service across US and European markets. He has been there the better part of seven years, has been a test engineer since 2013, and says he built the QA department, which did not exist when he arrived. For quality management and information security he says the team started them, not that he did alone.

A Computer Cannot Be Accountable

He shows what he calls a manual from IBM, dated 1979, carrying the line that a computer can never be held accountable and therefore must never make a management decision. He sources it to LinkedIn, saying the audience has probably seen it circulating once or twice.

He uses it to make the ownership argument for agent-written code. Never before, he says, has the implementation been more out of our hands, because the code arrives from agents rather than from a person you can question.

His grounding principle is that software exists to solve a problem in the real or digital world, and either way it affects people and it affects business.

From that he derives the question the rest of the talk answers: without knowing how it is working, how can anyone be sure the increment still solves the problem it was designed for?

The 10x Thought Experiment

He sets up an explicitly hypothetical scenario, premised on a group he does not endorse. Assume all the LinkedIn marketers are right and code output becomes ten, fifteen or fifty times what it was. What breaks first?

He offers three candidates: is the building the issue, is the proving the issue, or is the deciding the issue?

He never picks one. The sentence that would name the answer collapses mid-thought and he moves on, so the three-way question is posed and left open rather than resolved.

Where he does land is on scarcity. The QA job is not only quality control but supplying trustworthy evidence and decision templates that determine whether the product ships or does not ship.

He connects that to the traditional role. The test manager historically made sure the right person spoke to the right other person at the right time about the right thing, and that brokering function matters more once the pipeline is automated, not less. Its purpose is removing bottlenecks and misunderstandings before they reach the ship decision.

Note

Note: Classify the red result before you call it a bug. Try TestMu AI now!

Learning And The Job Market

His organisation is now more interested in the act of learning itself, meaning the definition of right from wrong and the measure of response: whether a defect is critical, and how that judgement feeds back.

He characterises the AI-and-jobs debate as binary, the machine either wiping out the work or leaving employment largely intact, and says the evidence points toward largely untouched. He names no study, author or institution, and hedges the appeal itself.

The shift he claims is compositional. Low-discretion tasks that used to be junior or training work are easiest to automate, while roles built on learning, decision-making and human judgement are still being created.

He is emphatic that learning is not a soft skill in the classical sense. It is the operational layer of decision-making: framing a problem, weighing competing constraints, assessing risk, and taking responsibility for outcomes, especially where the answer is not obvious.

Where AI falls short is precisely there, because complex systems do not give definite answers. You can encode principles like DRY and SOLID, security constraints and testability into scaffold rules, and the real system remains heterogeneous and wired to third parties and change.

He cites job-posting statistics to support the point, covering how sharply demand for judgement has risen and what share of postings ask for it. No dataset, region, time window or job board is named, and the framing sentence is damaged in the recording to the point where even the time direction is unclear, so the figures are left out here.

The Scaffold As Governance

He defines the scaffold, which he also calls the swarm or the agent cloud, as the governance backbone. It reviews intent and converts it into bounded work.

Each agent has specialised context and exchanges artifacts with the others, and findings are classified before they become something to repair or something to learn. He notes that sorting is necessary and does not resolve automatically into one or the other.

The stated ambition is a system that improves with experience: confront it with critique and confidence scoring so the scores rise, and give it the ability to self-doubt while developing.

He names the constraint that makes this hard, which is doing it without poisoning the context window by focusing on errors. A system fed only its own failures degrades.

The judgement he keeps for the system’s operators is the one you make with a junior engineer: deciding what is a valid lesson worth keeping, and what was a one-time incident that need not recur.

The QA Agent Chain

The QA agents run in parallel across a chain covering shaping, enabling, building and challenging, then validating and learning. The sentence where he lists it is garbled, though the steps are recoverable from the walkthrough he gives afterwards.

His non-negotiable rule is that every step between those agents has a scope, evidence, requirements, and, he stresses, stop conditions.

The reason for stop conditions is misattribution. When an issue surfaces he wants certainty it is an actual issue rather than a one-time incident or a test system mid-deployment, either of which can make a service look unstable.

Parallelisation only pays off, he says, if the scope, the ownership and the evidence travel with the work and are inherited along with it.

The resulting metrics are multivariate, and he reaches for a physics joke: they are like Schrödinger’s test results, true, and also capable of being untrue at a later point once the system changes.

He walks the chain through a single button as a verbal example. Shape gives the button a business case. Enable asks where it is used, whether it was used before, what buttons should and should not do, and whether a design reference exists. Build writes the code while simultaneously asking how it can be tested and what a successful test looks like. Validate and learn asks how effective the progress was, whether the metrics are still useful, and whether the lesson generalises.

In the build step he draws a distinction most suites collapse. Not every negative result is the same, and a browser crashing has nothing to do with your website technically speaking, so identical red results can have entirely different causes.

Test infrastructure that does not break, from TestMu AI

A Separate Challenge Path

The structural move he cares about is that QA adds a separately derived challenge path, keeping the builder context apart from the challenge context by design.

He anchors it in a familiar room. In a scrum meeting developers think about how to build it, and testers think about how to challenge it, whether it is testable, whether it is good, and when it is good.

His word is challenge, not break, which matters for a talk whose thesis is that QA validates intent. The published chapter list reframes it as testers asking how to break it.

The goal is independent convergence. The QA agents should reach their conclusions separately and, ideally, arrive at the same place as the builder path, exchanging information only afterwards.

He uses the tester-walks-into-a-bar joke to make the point about adversarial coverage, ordering one beer, two beers, a huge number of beers, a negative number of beers, then a lizard. The punchline is the case nobody tested, when a real person walks in and asks something ordinary and the bar burns down. His telling is scrambled in the recording, so this is the standard version.

Inside the validation cluster he counts six QA capabilities, which he also calls personalities, working on questions such as how to review the test report, what is known about the feature, what a validation actually is, and whether the intent is good. He never names all six. The cluster is explicitly recursive: it validates the validation process and challenges the review itself before anything leaves.

Red Is A Classification Problem

A red test result is a classification problem rather than a bug, because you do not know whether the use case or the circumstances were right in the first place.

Validation runs against a contract rather than against code. The system is not validating strictly against code, it is validating the intent, in a model he compares to contract-driven testing. That contract has to specify what is being validated against, what counts as an oracle, what is observed, and which actions are allowed.

Classification of a red resultWhat happens next
ProductHandled in the loop: reevaluated, rewritten, retested, with a test or a bug reported
BehaviourHandled in the loop
SpecificationHandled in the loop
Test issueHandled in the loop; his example is a test that was not good enough or was too flaky
Infrastructure failureHandled in the loop
Harness failureHandled in the loop
UnknownStops the chain and requires a human decision
Comma

That human then makes a second decision, which is whether the whole episode is a lesson worth learning or something that should not go back into the project’s consciousness at all.

Git Only Looks Backwards

He spends the last technical stretch on the failure mode of his own architecture, acknowledging it took almost twenty minutes to present an abstract concept.

The problem is that repositories, where the markdown files, skills and code live, are optimised for preserved results, so by design Git looks backwards.

You therefore cannot hold everything in the repo. Not all specs, not all the intents, not the definition of done, because then you would have to stream the repo for changes.

The consequence he wants to avoid is deliberation churn, where every change throws the agents into deciding whether that change has any relevance to the task in hand.

His answer is to keep issue trackers and test tooling in the loop as event systems and event streams, providing a buffer that keeps the agents focused on the issue at hand rather than continuously re-evaluating changes.

He adds third-party dependencies as an unsolved variable, naming an ecommerce platform and a payment service provider as throwaway examples of vendors shipping on their own release cadence, which keeps getting faster. The system therefore has to be constantly re-fed intent, ownership, scope boundaries, dependencies and evidence.

His forward-looking claim is that the future challenge is convergence rather than speed, illustrated by a hypothetical of twenty developers in a large corporation working on the same or neighbouring systems.

System One, System Two

He closes on Kahneman’s Thinking, Fast and Slow. System one is fast, automatic, emotional, leans on mental shortcuts, exists to save energy, and is often described as lazy.

System two requires continuous conscious effort, logical reasoning and deep concentration. His test case is asking an audience what 47 times 47 is, where the first impulse is to reject the task because of the energy it demands.

His characterisation of what the book explains, framed around what separates humans from their predecessors and what made humans dominant, is his own gloss rather than the book’s argument.

The human advantage he draws out is deliberate switching between the two modes, plus scale. Our context and experience windows are, in his rhetorical phrasing, infinitely larger than the machine’s, which is why people notice when something is off.

He illustrates with an unnamed client who asked him to lift quality but could not define quality or what a critical error was, could not verbalise what good meant, and knew only the next five big release dates and that it should have AI in it. The story is single-sourced and the client is not identified.

His point is that eliciting a definition of good from an inarticulate stakeholder is the same skill as specifying for an agent. Combining that human faculty with the decision templates the agents produce is what lets him land the title: not replaced, but promoted to the pilot seat.

Q & A Session

The session ran to time and the host took a single question from an attendee.

  • How do you prevent QA validation becoming the main engineering bottleneck when artifact review volume increases tenfold?

    Mathias Strziga: The old understanding is that validation must be done by a human. What changes is that we define the parameters and constraints for a system that is constantly learning. We do not want a system doing the same task next week, because it needs to gain experience and be taught the concept of validation from the team. Think of an orchestra conductor, guiding without playing the validation. There is no way to humanly control every line of code and every output and every artifact. That reframes the question rather than solving it: he offers no ratio, queue metric, staffing model or throughput mechanism, and the tenfold premise in the question goes unaddressed.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.

Author

...

TestMu AI

Blogs: 232

  • Twitter
  • Linkedin

TestMu AI is World's First Full Stack AI Agentic Quality Engineering platform that empowers teams to test intelligently, smarter, and ship faster. Built for scale, it offers a full-stack testing cloud with 10K+ real devices and 3,000+ browsers. With AI-native test management, MCP servers, and agent-based automation, TestMu AI supports Selenium, Appium, Playwright, and all major frameworks. AI Agents like HyperExecute and KaneAI bring the power of AI and cloud into your software testing workflow, enabling seamless automation testing with 120+ integrations. TestMu AI Agents accelerate your testing throughout the entire SDLC, from test planning and authoring to automation, infrastructure, execution, RCA, and reporting.

Add to Google preferred sources

Summarise with AI

Copied to Clipboard!
...

3000+ Browsers. One Platform.

See exactly how your site performs everywhere.

Try it free
...

Write Tests in Plain English with KaneAI

Create, debug, and evolve tests using natural language.

Try for free

Did you find this page helpful?

More Related Blogs

TestMu AI forEnterprise

Get access to solutions built on Enterprise
grade security, privacy, & compliance

  • Advanced access controls
  • Advanced data retention rules
  • Advanced Local Testing
  • Premium Support options
  • Early access to beta features
  • Private Slack Channel
  • Unlimited Manual Accessibility DevTools Tests