Local Agentic Theory for Accessible Mobile Games [Testμ 2026]

AI Was Always in Games

The history he sketches is short and useful, mostly because it deflates the idea that any of this arrived recently.

He starts with Alan Turing raising the possibility of machines solving chess, decades before anyone could build one, then moves to the 1950s when games first ran on computers at all.

The example he lands on is Pac-Man, whose ghosts run a finite state machine: eat a power pellet and their state flips from chasing to fleeing, then back. Simple conditional logic, and the first game AI most people can name.

State machines have not gone anywhere, which is part of the point. The question is not whether to add intelligence to games, but where that intelligence now runs.

What Does On-Device Unlock?

A distributed AI system calls a model or an API upstream and waits for a result. Keeping inference local changes four things at once.

• Speed - inference without the round trip, which is the difference between a game that reacts and one that pauses.
• Privacy - data private to the device does not need to leave it, though he is careful to call this the promised future rather than a guarantee.
• It works anywhere - the subway case. A game needing remote inference stops on a lost connection; a local model does not.
• Personalisation - a game environment and an agent tuned to one player, without pushing that back upstream.

He adds a consequence aimed squarely at a testing audience: if inference is local, testing can be local too.

RL Is Not Agentic AI

He spends real time on this distinction because the two get used interchangeably, and they behave differently.

Reinforcement learning trains a system by repeated trial and error against an environment. His canonical case is a car that cannot drive a track until it has driven it hundreds of thousands of times, at which point it holds a model of that track and needs fewer iterations for the next one.

The AlphaGo and AlphaZero lineage sits here, extending through AlphaStar to the MuZero and EfficientZero series, where the point of EfficientZero is learning a game in an order of magnitude fewer iterations. He is explicit that none of it is agentic.

An agentic system does not learn the world it moves through. It reads its information space, reasons about the next action with a language or vision model, and executes through a tool call, which makes it task-based and reactive rather than trained.

He declines to declare a winner, noting there is genuine research disagreement, and only observes that agentic systems map more naturally onto how the real world is navigated.

Perceive, Predict, Decide, Act

His demonstration is a rebuilt Space Invaders, a game solved by reinforcement learning long ago, driven this time by an agentic solver.

Making it agentic meant decomposing play into discrete callable actions: the ship can evade, it can shoot, it can move left or right based on the trajectory of incoming bullets.

The loop then runs perceive, predict, decide, act. Perceive the environment, predict where a bullet is heading or where an alien can be hit, decide between avoiding and attacking, then commit to the action as its own step.

His contrast is with a deep Q network trained on an action-reward policy. The agentic version is more legible, because each step is something you can read in natural language rather than infer from a learned policy.

The Space Budget

He flags this whole section as relevant to anyone testing such a system, which is the bridge from game architecture to QA.

Model weights compete for memory first, and denser weights cost more, which is why a two-hundred-billion-parameter model is not going on a phone and probably should not need to.

Second is the compressed state history, the record of what the agent has already done. He singles this out as the important one, because that history is what the next decision is made from.

Third is the working set: what the agent is doing right now and the plan it is working through. Fourth, and smallest in his accounting, is rendering the game itself.

The Frame Budget

Typical device refresh rates give roughly 16 milliseconds per frame. Work that overruns that window shows up as jank.

The complication an agent adds is that its share of the frame is not fixed. Rendering, planning and a margin for smoothness are partitioned against each other, and the partition moves as the agent’s plan changes.

The Energy Budget

The third constraint is the one users feel last and complain about most.

His observation is that phones are not primarily designed or optimised for AI workloads, so an agent running continuously consumes real battery life, and noticeably more on mid-tier hardware.

The direction of travel is neural architectures and the AI-focused chips arriving in devices, which cut the energy cost because they are built for that workload, and hand some frame budget back in the process.

Negotiating All Three

The framing that makes the talk cohere is that space, time and energy are not three separate problems. Ignore any one of them and the system does not work, which makes it a single engineering trade-off.

He names the techniques rather than hand-waving at them: weighted soft constraints, where some limits flex as long as you have ranked which, plus anytime search, local search, minimum conflicts and Lagrangian penalties.

His example of ranking is deliberately extreme. At one percent battery, energy is the least flexible constraint in the system and everything else gives way to it.

Anytime search gets the clearest description: work within whatever processing budget the frame leaves, and if there is none, take an action anyway so the agent keeps moving.

Gaze as a Reactivity Layer

Gaze estimation has existed for a while and keeps improving, and he treats it as two things at once.

It is an accessibility mechanism, and it is a new kind of reactivity: a game that responds to where attention actually falls, effectively a heat map of interaction.

The architectural consequence is that the agent has to work alongside an onboard convolutional or transformer vision model reading gaze accurately, which means an ensemble running on the device rather than a single model, and pushes back on every budget above.

Limits of the Model

He is direct about capability. On the ARC-AGI-3 benchmark, models have not reached human intelligence, and in action spaces specifically they are still catching up.

That limit bites harder locally, because there is no cloud to draw on, and capability drops sharply with model size. Whatever intelligence you get is whatever you packed into the device.

His second architecture example is a crossword solver, framed as constraint minimisation. Filling confident words and backtracking over ones that stop fitting makes the remaining puzzle progressively easier, which is a graph problem, as he notes many agentic problems turn out to be.

Looking forward he points at generalist agents that need little environment-specific training, at real-time vision assistants that see and act, and at neurally generated games where the world itself is model output rather than authored content.

Active Accessibility Testing

The accessibility argument gets compressed by time, but it carries the strongest practical claim in the session.

Today, WCAG conformance is generally established through manual testing frameworks or from user reports after the fact. An agent that interacts with the device can run automated checks and, more usefully, find problems nobody wrote a check for.

His example is a focus trap: focus lands behind a dialog and there is no way back out. That is not a property of any single element, so a static scan does not see it.

The case is strongest exactly where static testing is weakest, on dynamic interfaces, since a violation produced by a state combination cannot reliably be predicted from the code that set it up.

He closes on what on-device agentic development still needs: agents that think fast enough, models that predict both gameplay and interaction well enough to catch accessibility issues, memory matched to the game, and a shared way to describe a game in the first place.

Q & A Session

Four audience questions made it into the time remaining.

• What is the best way to simulate accessibility conditions in a sandbox?

  Shafik Quoraishee: Build the violations deliberately. WCAG 2.2 lists them, so you can generate a test site that commits them on purpose and write a suite that tries to catch each one. Deliberately inaccessible reference sites already exist for this, and an equivalent app is now easy to produce with a coding assistant. For an estate with existing violations he recommends working with the dev team to isolate modules so they can be tested individually, and describes having built exactly that: module-by-module accessibility checks driven by an agent. For a sandbox specifically, a dev version of the site with either a static plugin scanning for violations or an interactive agent on a device simulator.

• Do agents make games accessible by testing them or by adapting play?

  Shafik Quoraishee: Both. On testing, an agentic framework can check conditions much as an integration test would, treating getting stuck as a violation in itself and recording focus order to expose sequences that make no sense. On gameplay, the agent returns natural-language feedback, so it can report that a game feels too easy, or that a character cannot progress because the dialogue does not hold together, which brings difficulty and play balance into scope alongside conformance.

• Should the loop run inference every frame, or only on meaningful state change?

  Shafik Quoraishee: Either, depending on what you are testing, and the trade-off is cost. Every frame is defensible for UX testing and expensive whether the model is local or remote. The alternative is triggering on meaningful change, which then has to be defined: game over, a new screen, a sudden win. His own view is that complete coverage eventually forces a vision-based system looking at the screen, because dynamic interfaces produce violations that only exist in the rendering.

• Gameplay is non-deterministic, so how does an agent reason reliably on device?

  Shafik Quoraishee: Determinism varies by game, and an open world demands more model complexity than a bounded one. The two approaches diverge here: later RL systems such as EfficientZero handle open-world scenarios with more flexibility, while an agentic system reacts to events as they occur, proposing an action per event and building an understanding of the world that reduces the uncertainty it has to act under. On device the ceiling is whatever the hardware allows, so the answer depends on models continuing to get more efficient.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.