AI in Mobile QA: What Actually Works Today? [Testμ 2026]

The Boundary Today

The panel is unusually direct about the boundary, so it is worth setting out before the detail.

From Research to Testing

Jan Schnyder and Nicolas Muntwyler met at university, both with machine learning backgrounds and both long-time gamers.

The lineage they point to is the research one. Gaming and AI have been close since the early days of the field, and the labs they were reading, DeepMind among them, worked on games for exactly that reason.

Their own demos played AAA titles previously considered out of reach and went up on YouTube. The commercial turn came from noticing the obvious adjacent use: an AI that can play a game can also test one.

That led to work with several of the largest PC and mobile studios, which is the vantage point the rest of the panel speaks from.

Fully Vision-Based

The architectural choice underneath everything else is that the agent looks at the screen and nothing else.

No accessibility IDs, no DOM tree. A live stream comes from the device, the system reasons about what the next action should be, and it issues taps, swipes, clicks and keyboard or mouse input.

The consequence Nicolas Muntwyler draws is that every game presents an identical interface to the agent. What the title is, and how it was built, stops mattering.

Difficulty then shifts entirely to the gameplay rather than the integration. Mobile titles are often simple UI navigation, which agents handle well, while complex 3D games remain harder.

AI Does Not Get Bored

Asked where agents genuinely beat people, Nicolas Muntwyler gives the least glamorous answer available, which is why it holds up.

Open this screen and confirm twenty elements are present is exactly the work that degrades when a person does it repeatedly, and exactly the work an agent runs identically every time.

The second advantage is parallelism. A full test plan can run across thousands of devices at once, which removes the staffing problem of scaling a QA team up for a release and back down afterwards.

Jan Schnyder frames the result as a change in team shape rather than a replacement. A smaller group of QA professionals directs, observes and controls a fleet of agents, and spends its own attention on play testing and feel.

The Part AI Cannot Judge

Justin Yuan raises the limit from the investor seat, and neither founder argues with it.

A game can function correctly, contain no bugs, and still fail. Whether it is fun, whether a player gets frustrated, whether anyone comes back, sits outside what an agent judges well today.

Jan Schnyder’s reasoning for why that gap persists is the sharpest line in the panel: games are one of the rare goods crafted exclusively for humans, so a human stays in the loop by construction.

Nicolas Muntwyler is careful about claiming otherwise, noting there is no clean quantitative measure here. What customers do is compare agent judgements against the feedback they already have from their own player base, and on simpler questions, such as whether a feature is completable without too many hard stops, the data suggests the agent can call it.

Persona-Driven Play

The nearest thing to closing that gap is instructing the agent to adopt a persona.

Told to use the app as a 70-year-old would, or as a 10-year-old would, the agent’s play changes, and the emergent differences are what customers find worth having in the data.

Jan Schnyder presents this as early signs rather than a solved problem, and still lands on human review at the end for anything made for humans.

Verification and Discovery

The clearest structural idea in the session is the split between two kinds of testing, which behave differently and produce different evidence.

• Verification testing - the regression case, where you want to know whether it worked or found a bug. The high-level metric is exactly that, with the sequence that led to the failure shown underneath.
• Discovery testing - the newer capability. You tell the agent you have shipped a feature and ask it to explore and try to break it, and it returns the specific example, what led up to it, and reproduction steps for a human to confirm.

What comes back from either is more than a report. Jan Schnyder points to a full recording plus every action and the reasoning behind it, which is available for sentiment analysis or whatever else the team wants to do with it.

That is a real difference from traditional automation, where a failure gives you a stack trace and a screenshot rather than an account of what the runner was trying to achieve.

Why Was QA the Beachhead?

Justin Yuan’s firm invests where AI meets games, and his account of why QA came first is about willingness rather than technology.

Across years of conversations with publishers and developers he describes a natural hierarchy of AI adoption, in which testing is the low-hanging fruit: a permanent problem that studios were already prepared to point AI at.

The second half of his thesis is what the beachhead opens onto. What a fleet of agents operating real devices provides is infrastructure, and infrastructure travels further than the first use case.

Beyond Bug Finding

The adjacent uses the panel names are the most concrete evidence that the infrastructure argument holds.

• Monetization verification and ad auditing - checking that the commercial surfaces behave as intended at scale.
• Age verification - work Justin Yuan describes with another portfolio company, aimed at identity and verification for children across games and social apps.
• Compliance testing - which he expects to move from a do-it-if-you-must obligation to a front-and-centre concern in product design, following high-profile fines in the hundreds of millions.
• Accessibility testing - a natural fit precisely because the agent does not get bored, moving it from a final check to something running from the start of a sprint.
• Competitor analysis and market research - running agents across rival apps to gather structured insight, with the data returned automatically rather than through a scraper someone had to write.

The unplanned one they mention is ordering food delivery through the agent, which sounds like a joke until you notice what it demonstrates: the product is a natural-language interface to a device, and testing is the first thing people point it at rather than the only thing.

Jan Schnyder also draws the maintenance contrast directly. Scripted mobile automation is brittle and consumes real resources to keep alive, whereas an agent adapts to a changed menu or environment without a maintenance pass, which is what lowers the entry barrier.

Why Does Black Box Beat Injection?

Asked why they stayed outside the app rather than hooking into it, Nicolas Muntwyler gives two reasons, and the first is a genuinely good testing argument.

A human looks at a device. They do not inject themselves into the application. So anything visible from that viewpoint should be catchable as a bug.

His example makes it concrete: a button that is invisible but still pressable. Click it through an accessibility ID and the test passes. Look at the screen as a person would and it is obviously a defect that should be flagged.

The second reason is adoption. There is no integration to build. Supply a build, or a URL for a website, describe what the agent should do, and that is the setup.

Context is layered on top through a knowledge base holding design and reference documents, and through an MCP integration that lets teams author test cases from inside their own stack. For verification tests a human authors the cases; for discovery the agent needs far less.

On the obvious worry that a vision agent just clicks around, Nicolas Muntwyler is clear that these runs are not monkey testing. The agent reasons about what would make a sensible test scenario and then works out the buttons on the way to the goal.

Testing Became the Bottleneck

The observation that most deserves to travel beyond gaming is about where AI moved the constraint.

Across the studios they work with, creation got dramatically faster. Verification did not keep pace, and it still has to happen before release, so testing is now the thing releases wait on.

Their answer is to push it left into the development loop. A GitHub hook lets a developer pushing a feature trigger a run that spins up a device, deploys the build, and exercises it at pixel level rather than at code level, reporting straight back.

Justin Yuan adds the industry context that makes this urgent in games specifically: development cadence in consumer entertainment is already rapid and increasingly personalised, with a whole discipline of live operations built around it.

Q & A Session

The audience engage panel fed two questions into the session, and the host closed with one of his own to all three panellists.

• How does the agent decide its execution path, and what happens to coverage if it misses one?

  Jan Schnyder: That is what verification tests are for. Alongside the execution instruction, such as play the tutorial or complete the signup flow, you define expected results to be checked explicitly: does this popup appear, does the green banner show on signup. The agent verifies them one by one and flags a mismatch to a human reviewer, who confirms it and routes it to the developers or the test management system.

• What does the QA role look like now?

  Nicolas Muntwyler: It gets bigger rather than smaller. Where a tester previously owned a set of cases they executed or scripted, the same person now gets roughly ten times as much done and moves into QA management, taking ownership of quality across far more of the app. We see people progressing up the responsibility ladder faster as a result.

  Justin Yuan: I see the same pattern across the agentic ecosystem generally. The job changes into something closer to agent supervision, managing thousands of concurrent runs and specifying intent precisely enough for the system to act on. The residual human skill goes where the bottleneck is, which for now is still judging whether an experience is enjoyable.

• What would you caution a founder building in AI today?

  Justin Yuan: Ask whether what you are building is a feature the foundation models or an incumbent will logically absorb, or a standalone product with a reason to exist as its own business.

  Jan Schnyder: The barrier to building has never been lower, which is exactly the problem, since anyone can now build something. The most valuable trait available is agency: identifying the thing that matters, then staying persistent while adapting to a field that keeps changing underneath you.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.