The Future of Portable AI Applications [Testμ 2026]

A Harder Portability Problem

Prashant Sharma opens on the one phrase every developer has heard, and traces the ladder of attempts to kill it: operating systems, abstraction layers, virtual machines and the web, each generation moving closer to the idea that developers should spend time building apps rather than worrying about every piece of hardware underneath.

His framing of the moment is that AI forces the industry to solve the same problem again, and this time it may be considerably harder. Distributed computing has been getting steadily more difficult across the roughly 10 to 20 years since AWS, now compounded by the proliferation of models.

He inverted normal presentation style deliberately, trying not to go beyond five words on any slide so the time would go into discussion rather than definitive answers. He says up front that he probably will not have all of them.

The host, Kavya, Director of Product Marketing at TestMu AI, set the same problem commercially: most AI applications built today are tightly tied to a model provider, a runtime, a cloud and a chip. A reasonable trade when moving fast, and a hard one to unwind 18 months later when the economics of the hardware change.

AI Beyond The Data Center

AI is escaping the data center onto laptops, phones and the browser, into vehicles, cameras, robots, wearables, industrial systems and increasingly edge devices.

He expects the next generation of devices to be ambient computing, looking extremely different from anything we are used to today.

That creates a developer expectation he states three ways: not rewrite it, not rebuild the entire software stack, not optimise it manually for 20 different processors. Build it once and run it everywhere, which is what people call portable AI.

Then he splits the room, and this distinction is the spine of the talk. A developer asks whether their application can run everywhere. A tester has to ask the much harder question: if it runs everywhere, does it still work everywhere?

The Compatibility Matrix

He recounts a conversation with someone at a frontier lab whose job is making their coding assistant work across all platforms and all devices, including emerging categories. That person described the problem to him as a grid.

He is transparent that the matrix visual on his slide is AI generated, while insisting the matrix itself is not uncommon and is what many teams struggle with today.

The example is deliberately simple: a voice assistant, an image recognition application, or a small language model embedded inside a productivity application. You train it, you test it, everything looks great, and then you have to deploy it.

The targets he lists are the cloud, a Windows laptop, a Linux box, an Android phone, a browser and an embedded device.

The resulting matrix multiplies model by framework by runtime by operating system by CPU, GPU, NPU or accelerator by precision format by memory constraint by device. He calls it a pretty intimidating test matrix.

Portability Beyond Pass/Fail

Historically, software portability was largely binary. Does the application run, yes or no. AI introduces a far less controllable middle ground.

His illustration of that middle ground: the application runs, it does not crash, the model loads, the inference completes, and projected throughput drops from 600 tokens a second to 15. Those numbers are a hypothetical about an unoptimised deployment rather than an observed benchmark.

In the classical sense the application is running, and it is not optimised for the hardware in experience terms. Every functional test passes and the experience may still be wrong.

Behind a passing test he lists the failure modes that hide: memory consumption doubles, the device starts thermal throttling, optimisation changes behaviour.

So “does it run?” is no longer a binary answer, and “does it run with the experience we intended?” is an entirely different testing problem.

The smartwatch story makes it concrete. A smartwatch company’s route and map tracking is completely off because of a roughly 300-millisecond gap between the moment it picks up the GPS signal and the moment it encodes that on the hardware. He never names the company and hedges the story himself as famous or infamous depending on who you ask.

Same Model, Different Answers

He asks the audience to imagine running roughly the same open-weight model on a high-performance cloud cluster with an accelerator, then a laptop GPU, then an NPU, then a smartphone, then an embedded device.

Conceptually you would expect the same results. In practice things get a lot more convoluted.

The reasons he lists are four: the underlying hardware is different, the available operators may be different, the runtime implementation may be different, and numerical precision and memory behaviour differ.

The consequence is that what looks like a single monolithic AI application is actually multiple execution environments, so testing has to look beyond simple functional correctness.

The questions it must answer instead: did model quality change, did latency change, did memory consumption change, and can the workload sustain its peak performance for an extended period.

The Acceptable Experience Bar

Portability does not necessarily mean identical, because different targets optimise for different things.

So the goal is not whether AI can behave identically everywhere. It is whether AI can deliver an acceptable experience everywhere.

That reframing puts a requirement on tooling: testing has to understand the context it is operating in, not just the assertion it is checking.

Making Hardware Boring

Hardware should almost become boring, he argues, and notes the irony given his own background running from distributed systems, operating systems and databases through to silicon architecture. He works for Arm, so this account of an emerging abstraction stack is his industry view from inside that layer rather than a neutral survey.

The stack he describes has the application and model at the top, then portable runtimes, then optimisation layers, then CPU IP classes, GPU, NPU and specialised accelerators.

The developer should not have to manually orchestrate every thread. The runtime and compiler ecosystem should increasingly determine what to execute, in what precision format, how memory is managed, what leaks, and how efficiently the workload runs.

His analogy is that most web developers do not spend their morning asking which CPU instructions their JavaScript executes. They build the experience and the platform handles the complexity underneath.

This is not because hardware is unimportant. It is quite the opposite, since hardware has become increasingly sophisticated, and the developer just should not have to think about it every few minutes.

The trade-off he names explicitly is the one testers should hold onto: the more complexity you hide from developers, the more responsibility moves into validation, because something still has to verify that all those layers work together.

The 91% Accuracy Question

Traditional correctness is input, expected output, compare the two, pass or fail. AI complicates that model significantly.

His worked example is a hypothetical image classifier rather than a measured model. The cloud version gives 94% accuracy, and after optimising for an edge device it drops to roughly 93.8%, a figure he approximates on air. Did that test pass? Probably.

Push it to 91%. Did it pass? Maybe. Now suppose 91% but inferencing is four times faster and uses half the power. There is no single answer to whether 91% is a pass or a fail.

What is really being evaluated is a multi-dimensional quality envelope, which for portable AI should include at least five things: correctness, performance, efficiency, quality and, most importantly, portability.

The open question he leaves with the room is whether you can hold those characteristics across all the devices you care about.

Testing As An Experiment Loop

Testing moves from something that happens at the end of development to something much closer to a continuous experimentation loop. Every model optimisation is an experiment, every hardware target is an experiment, every runtime upgrade is an experiment, and every quantisation strategy is an experiment.

The question each experiment must answer is whether the experience got better, or whether something broke without anyone noticing.

You cannot test that matrix manually. Multiply models, frameworks, runtimes, operating systems, CPU, GPU and NPU targets, precision formats, devices and versions and the number becomes absurd quickly. He cites a conversation with someone at a premier hardware silicon company who was tracking almost 164 different versions of these parameters, a second-hand figure from an unnamed source.

So testing itself has to become portable and automated. When a developer commits a new version, the system deploys the application across representative hardware targets, whether simulated, real or hybrid, a cloud instance, an NPU, a smartphone, whatever device you pick.

It then automatically measures accuracy, memory, latency, power and thermal throttling, and reports in trade-off terms. His sample outputs, invented for illustration, read like “model quality has dropped 2.3% on the target”, “latency has increased 40% after the runtime update”, or “this quantised model meets your accuracy requirements and reduces memory consumption”.

The evolution he draws from that is a move from “did it pass?” to “is this the best version of the application for this target environment?”

Developer Intent As The Spec

For years software testing tried to abstract hardware away, and he did it himself as an active developer. AI now forces hardware back into the conversation, not for the application developer but for the testing infrastructure.

Two systems running the same model can produce dramatically different user experiences, and a benchmark measured on a server may tell you very little about what happens inside a phone, or the reverse.

Duration matters as much as the metric. A five-minute test may tell you very little about thermal behaviour compared with running the workload for six hours on battery, and a latency benchmark may hide memory pressure.

The design goal he states is that AI software optimisation must be hardware-aware without forcing every developer to become a hardware expert. Nobody should have to be one unless it is their choice.

The division of labour follows. The developer defines the experience they need, for example inference under 100 milliseconds, memory below a certain threshold, power within an acceptable range, and the testing infrastructure determines whether that experience can be delivered on the target hardware.

The Build-Once Mental Model

He is blunt that build once, run everywhere does not mean the same binary magically executing identically on every device on earth. He calls that the wrong mental model, and says it is already shifting.

What it does mean is a chain: the developer builds an application around an AI capability, the underlying software stack adapts that capability to the available compute, the runtime decides how to execute it, the compiler optimises it, the hardware accelerates it, and automated testing continually verifies the application stays inside the quality envelope. That envelope is developer-defined, not user-defined.

The payoff that excites him is a change in the question developers ask. They stop asking which accelerator the device has, and start asking what experience they can offer to this form factor, this hardware, this target user.

His historical framing is that the history of computing is in many ways the history of creating abstractions that let developers build bigger things without understanding the complexity underneath, and AI is now entering that abstraction layer.

Q & A Session

The chat supplied the first two questions. The third was one the host said she keeps encountering herself and had been having a lot of conversations about, rather than an audience submission.

• If an AI agent can make decisions faster than humans can review them, does human in the loop still work at enterprise scale?

  Prashant Sharma: Yes for the mid term. A lot of enterprise decision-making is intuitively embedded in the people operating the systems, so when people ask when a large enterprise application will be fully vibe coded, the answer is not only about the application and its interconnectedness. It is about the intuitive human knowledge of how systems work, which he called phenomenally important. He expects the human-in-the-loop dependency to start reducing, and says a reduction is already visible, but for the foreseeable future he does not see AI replacing humans completely.

• How would you build an agent that is context and capability aware, and would that solve some of the optimisation issues across systems and devices?

  Prashant Sharma: Taking the second half first, it perhaps can, and in fact already does today, beautifully, in scenarios that are far more predictable and have enough historical precedent. Agents can simulate optimisation issues across devices that are two, three or four generations into production at some scale. To build one, he would sandbox it against emerging use cases, watch how it behaves, then iterate. He said mobile hardware makers experiment this way for device generations going back four, five or six years, and gave the example of a game studio asking a phone company for backward compatibility across the last five generations of handsets. Agents can do that benchmarking and optimisation heavy lifting; it does not solve all the problems, but it minimises them.

• If AI continues improving at its current pace, what will change most in software engineering over the next 5 to 10 years, and what should engineers start doing today?

  Prashant Sharma: The systemic change is already visible: mundane software development is no longer required, and production-grade software he is aware of, the regular routines, is already being written by machines and getting better over time. Critical thinking is not a skill that will go away, which he compared to still learning languages at college even when you already speak several. Asked by a family friend’s child, newly accepted to an undergraduate software engineering course, whether it was still worth studying, he said yes, for the critical thinking and systems-level thinking the discipline trains. He noted he once wrote code for 15 hours straight and no longer does, but still writes the critical piece himself and lets the machine do the heavy lifting. Ten years, he said, is too long a horizon to call.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.