Mobile App Testing: The Complete 2026 Guide

Mobile app testing is the process of verifying that an Android or iOS app installs, works, performs, and stays secure on the real devices, OS versions, and networks your users have. It combines functional checks, such as sign-in, checkout, and navigation, with non-functional ones, such as speed, battery, security, and accessibility, run manually and with automation across a device matrix.

The hard part is not writing tests. It is coverage: Android holds 69.17% of the global mobile OS market against iOS at 30.81% in StatCounter's September 2026 data, Android fragments across thousands of models and vendor skins, and a build that passes on an emulator can still crash on a three-year-old handset with 3 GB of RAM.

What is Mobile App Testing?

Mobile app testing is quality assurance for software that runs on phones and tablets. It answers four questions before release: does every feature work, does it work on every device in scope, does it hold up under real-world conditions such as a bad network, a low battery, and interruptions, and is user data safe.

Mobile App Testing vs Mobile Application Testing vs Mobile Testing

"Mobile app testing" and "mobile application testing" mean the same thing; the second is the formal term used in RFPs and test plans. Mobile testing is wider: it also covers mobile websites in Chrome and Safari and device-level behaviour such as battery drain and carrier networks.

Types of Mobile Apps and What Each Needs Tested

Mobile apps come in four types: native apps built for a single platform, hybrid apps wrapping web code inside a native shell, cross-platform apps sharing one codebase across both, and mobile web apps opened in a browser. Each needs a different matrix.

Which one you are shipping changes what you have to test:

For the build-side trade-offs behind that choice, see this comparison of web vs hybrid vs native apps.

The Mobile App Testing Process in 9 Steps

The mobile app testing process runs in nine steps: plan, choose devices, write cases, test manually, automate, run usability and beta, test performance, test security, then gate the release on real devices.

Steps 1 to 3 happen before anyone touches a phone; skipping them is why suites end up covering whatever was easiest to script.

• Write the test plan. Scope, platforms, OS floor, entry and exit criteria, owners. Agree what "done" means for this release before testing starts.
• Build the device matrix. Pull device and OS distribution from your analytics, take models until you hit 80% of installs, then add the risk edges: a low-RAM Android, a foldable, a tablet, and one Samsung One UI or Xiaomi HyperOS device. Current and previous iOS cover almost all iPhones: Apple reports that 79% of all iPhones run iOS 26, rising to 86% of iPhones introduced in the last four years, with a further 14% on iOS 18, as measured by devices that transacted on the App Store on 7 June 2026. Review quarterly.
• Write test cases. Functional cases from requirements and user stories; non-functional cases for performance budgets, security controls, and accessibility. AI agents such as KaneAI can draft cases from PRDs, Jira tickets, and recordings.
• Test manually first. Exploratory sessions on real devices for every new feature. Log device, OS, build, and a screen recording with each defect.
• Automate what has stopped changing. Regression and smoke flows go into Espresso, XCUITest, Appium, or Maestro and run in parallel across the matrix on every build.
• Usability and beta. Moderated sessions with 5 to 8 users per round; TestFlight on iOS, and Google Play internal or closed tracks on Android, for beta.
• Performance. Cold start, frame rate, memory over long sessions, battery and thermal behaviour, on a low-end device as well as a flagship.
• Security. Token storage in Keychain and Keystore, TLS everywhere, certificate validation, session expiry, permissions the app does not need. Use OWASP MASVS as the checklist.
• Release gate. Full regression plus the release checklist below on real devices in the matrix. Fix, re-run, ship.

How Do You Choose Which Devices to Test On?

Build a device matrix from your own install-base analytics: cover the devices that make up the bulk of your installs, then add a low-end handset, a tablet, and one vendor skin. Review it quarterly.

You cannot test every device, and you do not need to. What you need is a short, defensible list covering most real users and the riskiest edges.

Start from your analytics, never a generic "top phones" list. Your install base is the only data that says which devices actually matter.

Build the Matrix in Four Passes

Each pass adds a different layer of coverage:

• Pass 1, coverage - Sort active devices by user count, taking models from the top until you reach 80% of installs.
• Pass 2, OS spread - Add the current and previous iOS version. Android needs the newest release plus two or three before it.
• Pass 3, risk edges - Add a low-RAM Android device, a foldable or large screen, and one tablet.
• Pass 4, vendor skins - Add a One UI, HyperOS, or ColorOS device beside stock Android. Permission and background limits differ.

On the adoption figures in step 2 above, the current and previous iOS release cover almost every iPhone user. Android fragments far wider, so it needs the extra spread.

For most teams this lands between 8 and 15 combinations per release, with a wider matrix reserved for major launches.

Every team I have sat with expected that number to be much larger before they actually pulled their analytics.

A Worked Example

Here is a matrix for a consumer app with an even Android and iOS split. Swap the models for whatever your analytics show.

Review the matrix once a quarter. Device usage shifts, and a list built two years ago stops matching your users.

When you need broader coverage than an in-house lab can hold, a real device cloud gives you the same devices on demand without procurement.

10 Types of Mobile App Testing

Ten types of mobile app testing matter: functional, performance, compatibility, security, usability, interrupt, network condition, localization, regression, and accessibility testing. They run at different cadences: functional and regression checks on every build, compatibility and performance against the device matrix each release, and security, localization, and accessibility audits once per milestone.

Functional Testing

Functional testing verifies that every feature behaves the way its specification says it should, covering user interactions, navigation flows, form inputs, API responses, and data processing.

It catches broken buttons, validation that accepts bad input, logins that fail after a token refresh, and checkouts that stall mid-payment.

Most teams script the critical paths, install, sign up, search, and pay, then run them on every build and keep exploratory functional testing for the edge cases a script never reaches.

Performance Testing

Performance testing measures how fast the app launches, how smoothly it responds under load, and how stable it stays when memory, CPU, and battery are constrained.

It catches cold starts that creep past a couple of seconds, list scrolling that drops frames, memory leaks that crash long sessions, and background work that drains the battery.

Run it on real hardware and on low-end handsets as well as flagships, because mobile performance testing figures from an emulator describe the host machine, not the phone.

Compatibility Testing

Compatibility testing confirms the app looks and behaves consistently across device models, screen sizes, OS versions, and vendor skins. It catches layouts that break on a foldable, permission prompts that changed between OS releases, and translated labels that overflow a button on a small screen.

Coverage is a matrix problem rather than a device count: a team that owns ten handsets tests a fraction of the models and OS versions its users run, which is where a cloud of real devices fills the gap.

The basics of compatibility testing apply here.

Security Testing

Security testing finds the weaknesses that expose user data: weak or missing encryption, tokens stored in plain text, sessions that survive logout, unprotected API endpoints, and permissions the app never needed.

It also covers what regulation demands, including GDPR and HIPAA obligations around data handling and consent. Run static analysis on every build and a deeper penetration pass each release.

The security testing findings that matter most usually sit in the network and storage layers rather than in the UI.

Usability Testing

Usability testing evaluates whether a real person can finish the app's core tasks without being taught how, covering navigation logic, information hierarchy, gesture support, and error recovery.

It catches the problems no assertion can express: a checkout step nobody finds, a swipe that fights the operating system's own back gesture, an error message that quotes a code instead of a fix.

Run moderated sessions on real devices with a handful of participants per round and repeat after each redesign, because usability testing findings expire as soon as the flow changes.

Interrupt Testing

Interrupt testing checks how the app survives events that arrive mid-session: an incoming call, a push notification, an alarm, a low-battery warning, a network switch, or the user backgrounding the app and returning an hour later.

What it catches is lost state, a half-filled form wiped, an upload abandoned silently, a crash on resume. Fire each interruption at the least convenient moment, mid-transaction rather than on an idle screen, and confirm the app returns the user exactly where they left off.

Network Condition Testing

Network condition testing validates behaviour across 5G, 4G, 3G, congested public WiFi, and full offline mode, including recovery when the connection drops mid-request.

It catches blocking calls that freeze the interface on a slow link, retries that duplicate a payment, caches that serve stale data, and error screens with no way back.

Throttle deliberately instead of waiting for a bad day: run the critical flows on a simulated 3G profile and in airplane mode, and confirm queued actions resend once the connection returns.

Localization Testing

Localization testing confirms the app adapts correctly to other languages and regions, covering translated strings, date and number formats, currency, time zones, and right-to-left layouts for languages such as Arabic and Hebrew.

It catches text that overflows its button once translated into German, dates that read as the wrong month, and hard-coded strings that never reached the translator. Set the device locale rather than relying on an in-app language switch alone.

Our guide on how to perform localization testing covers formats and right-to-left support.

Regression Testing

Regression testing verifies that new code has not broken functionality that already worked, re-running a stable suite against each build so a fix in one flow does not break another.

It catches what a release cycle reintroduces: a permission prompt that reappears, a payment path that breaks after an SDK upgrade, a screen that stops rendering on an older OS version.

Automate the suite and prune it without sentiment, because a regression testing pack nobody trusts is a pack everybody skips.

Accessibility Testing

Accessibility testing checks the app against WCAG criteria and the platform accessibility APIs, which matters both for ADA compliance and for users who navigate without sight or fine touch precision.

It catches unlabelled icons, focus order that jumps around the screen, contrast below threshold, and tap targets too small to hit reliably.

Run it with TalkBack on Android and VoiceOver on iOS, then verify every interactive element is announced, that focus order follows the visual order, and that tap targets stay usable at the largest system font size.

What Is the Difference Between an Emulator, Simulator, and Real Device?

An emulator imitates Android hardware, a simulator imitates the iOS environment, and a real device is the phone itself. Use virtual devices for fast feedback on every commit; use real devices for anything touching hardware, battery, sensors or the real network, and for the release gate.

Emulator-based app testing

An emulator is a virtual platform that duplicates an Android handset on your computer. You can test APK files without physical hardware using an apk emulator online.

Simulator-based app testing

A simulator reproduces the iOS environment on your machine and runs the app against a chosen Apple device profile, with no physical hardware required.

Because it borrows the host operating system rather than emulating silicon, it starts fast. That speed is what makes iOS app testing practical during development.

Real device-based app testing

Real devices run your app on the same hardware your users hold. They surface defects that virtual environments hide, including thermal throttling, battery drain, and sensor drift.

I have watched a suite go fully green on simulators and still ship a crash that only appeared on a three-year-old handset with 3 GB of RAM.

The rule that follows from the table: anything that touches physical hardware, battery, or the real network needs a real device before release.

If you only need a quick look at how a build renders on Android without installing anything locally, an Android emulator online covers that in a browser.

Manual vs Automated Mobile App Testing

Manual and automated mobile app testing cover different halves of the same matrix. Manual testing owns judgment work: exploratory sessions, usability, visual polish, and bug reproduction on a feature that is still changing.

Automated mobile app testing replays fixed flows with Appium, Espresso, XCUITest, or Detox, so it owns regression, cross-device compatibility, and every check that runs on each build.

A test that repeats often, spans many devices, and covers a settled flow belongs in automation; a one-off check on a moving flow stays manual.

A practical rule follows from that split: start a new feature with manual exploratory testing on real devices, then automate the stable paths once they stop changing.

Because both approaches run against the same real hardware, a bug found by hand can be handed straight to automation as a regression test once the behaviour is understood.

Splitting the work between the two approaches is only part of it, because the tests still have to be authored, evidenced, and maintained.

In this TestMu Conf 2026 session, Shravan Mahajan runs that whole loop end to end with kane-cli in The Full Agentic QA Loop for Mobile Applications: Author, Evidence, Assure, Maintain, covering how you author coverage, capture what actually happened, assure it against intent, and keep it from rotting as the product changes.

$ kane-cli generate "Login flow of a native Android banking app: email and password sign-in, biometric fallback to PIN, session restore after the app is backgrounded, and offline error handling" --agent --scenario-limit 4 --per-scenario-limit 3

{"type":"generate_done","request_id":"40889","status":"completed","scenario_count":4,"case_count":12,"refine_hint":"kane-cli generate \"<refinement>\" --refine --req 40889","save_hint":"kane-cli generate --save --req 40889"}

Mobile App Testing Checklist for 2026

A mobile app testing checklist is the pre-release pass a team runs on every build, covering install and update, core flows, interruptions, network conditions, performance, security, and accessibility. Run it on a real device from your matrix, then automate the items that repeat every release.

The line I add to every checklist is the permission-denied path, because apps that assume a granted permission fail silently.

• Install the signed release build with almost no free storage left, then uninstall and reinstall it.
• Cold-start a first launch on a throttled connection and confirm the splash screen resolves.
• Update from the currently published store version and confirm data, sessions, and settings survive.
• Complete signup, login, logout, password reset, and every social or SSO provider.
• Submit each form empty, at maximum length, and with special characters and emoji.
• Run purchase flows in Apple's StoreKit sandbox and Google Play's licence testing track, including refunds and renewals.
• Open every deep link and push notification with the app running, closed, and uninstalled.
• Press the Android system back gesture on every screen and confirm no path dead-ends.
• Fire a call, an alarm, and a notification mid-flow and confirm entered data survives.
• Background the app for five minutes and confirm the screen and session are restored.
• Rotate, split-screen, and raise the system font to its largest setting on every screen.
• Deny camera, location, and notification permissions, then grant them, and check both directions.
• Run the app under low battery, low memory, and battery-saver mode.
• Throttle the connection to 3G and confirm every screen shows a loading state.
• Drop to airplane mode mid-request and confirm a clear error and a retry path.
• Switch between WiFi and mobile data mid-session, and confirm offline mode, where supported, syncs queued actions on reconnect.
• Measure cold start on a low-end device against the previous release, and watch memory through extended scrolling.
• Confirm tokens live in Keychain or Keystore, all traffic uses HTTPS, and invalid certificates are rejected.
• Walk the app with VoiceOver and TalkBack and confirm every interactive element is announced.
• Disable animations on all test devices and emulators before an automated run.

For a fuller set of scenarios to turn into test cases, use this mobile development test case template.

How Do You Test the Hard Mobile Features?

Biometrics, camera, deep links, push notifications, and in-app purchases all need platform sandboxes or real hardware. Scripted UI tests on their own cannot reach any of them reliably.

Teams skip these features more than any others, and each has a specific way in:

Biometrics

Both platforms can simulate a fingerprint or face match on a virtual device, which covers the happy path. Enrolment, lockout after repeated failures, and fallback to passcode need real hardware.

Test the failure branch first. An app that grants access when biometric auth errors is a security defect, not a UX one.

Camera and QR Scanning

Emulators feed a synthetic image, so they prove the code path but not the experience. Focus speed, low light, and glare only show up on real devices.

Keep a printed test QR code at your desk. It reproduces scanning defects faster than any automated harness.

Deep Links

Android App Links and iOS Universal Links both depend on a verification file hosted on your domain, so they fail in staging while working in production, or the reverse.

Test three states for every link: app installed and running, app installed but closed, and app not installed at all. The third should route to the store, not a dead page.

Push Notifications

Operating system behaviour changes with app state, so test foreground, backgrounded, and fully closed separately. Verify the banner renders and the tap deep-links to the right screen.

The case most teams skip is permission denied. Confirm the app still functions and does not block a core flow waiting for a notification that will never arrive.

In-App Purchases

Use the platform sandboxes rather than live payment rails: StoreKit testing in Xcode for iOS, and licence testing on Google Play for Android.

Cover the whole lifecycle, not the purchase alone. Restore purchases, declined and cancelled transactions, subscription renewal, and revoking access after a refund all need explicit cases.

Mobile App Testing Tools and Frameworks

Pick the framework from your platforms, then the infrastructure from your device needs.

Two 2026 changes matter when you shortlist. Appium 3 is now the current stable major line, 3.8.0 as of October 2026, and it raises the Node floor to 20.19 while supporting only the even-numbered LTS lines. It also removes many previously deprecated server endpoints, most of which have direct replacements, so some Appium 2.x suites need small changes before they run on it. The project keeps an official Appium 2 to 3 migration guide for that move.

The second change is infrastructure. Google has deprecated Firebase Test Lab, which is fully shut down on September 30, 2027, and points teams to the Developer Device Platform on Google Cloud instead. That is far enough out to plan calmly, but it does rule Firebase Test Lab out for a suite you are starting now. Note that Android Device Streaming is not part of that shutdown.

How to Choose a Mobile App Testing Tool

Match the tool to your platforms, your team's languages, and the device coverage you need, and score candidates on language support, OS coverage, codeless authoring, and reporting depth.

Single-platform apps should start with the native framework, Espresso or XCUITest, because nothing else is as fast or as stable. Cross-platform apps should start with Appium, or Maestro if nobody wants to write code.

Once the framework is settled, the infrastructure question is only how many real devices you need and who pays for them. Configure the test environment before the first script is written.

Use this checklist when comparing candidates:

• Support for the programming languages your team already writes tests in.
• Codeless authoring for scenarios that repeat every release, which cuts the time spent rewriting them.
• Coverage of every operating system you ship on, including the older Android and iOS versions still in your install base.
• The ability to detect, record, play back, and automate every action in a test scenario.
• Reporting that logs a plan description, screenshots, and defect points for every run.

For the full comparison of supported frameworks, device coverage, and selection criteria, see our roundup of the best mobile app testing tools.

Mobile App Testing Frameworks

A framework is what you write tests in. The choice comes down to one question: are you shipping on one platform or two?

Native frameworks win on speed and stability. Cross-platform frameworks win when you cannot maintain two separate suites.

For a deeper, side-by-side comparison, this roundup of the best mobile app testing frameworks covers Espresso, UIAutomator2, XCUITest, Appium, Maestro, Detox, and Flutter's integration_test and Patrol across gray-box versus black-box tradeoffs.

Android Testing Frameworks

Android teams pick from these:

• Espresso - Google's native Android framework. It runs inside the app process, syncing with the UI thread and producing far less flakiness. More on Espresso testing.
• UIAutomator2 - Handles cross-app and system-level interactions such as the notification shade and permission dialogs. Espresso cannot leave your app; this can.
• Compose UI Test - The testing API for Jetpack Compose screens, used instead of Espresso view matchers once you migrate off XML layouts.

An Espresso test for a login flow is short, and it reads like the steps a manual tester would follow:

@RunWith(AndroidJUnit4::class)
class LoginTest {

    @get:Rule
    val activityRule = ActivityScenarioRule(LoginActivity::class.java)

    @Test
    fun validCredentials_navigateToHome() {
        onView(withId(R.id.email))
            .perform(typeText("qa@example.com"), closeSoftKeyboard())

        onView(withId(R.id.password))
            .perform(typeText("secret"), closeSoftKeyboard())

        onView(withId(R.id.login_button)).perform(click())

        onView(withId(R.id.home_title))
            .check(matches(isDisplayed()))
    }
}

iOS Testing Frameworks

Apple's frameworks ship with Xcode:

• XCUITest - Apple's native framework, built into Xcode. Tests run in Swift or Objective-C, with parallel execution across destinations. More on XCUITest testing.
• XCTest - The framework XCUITest builds on. Use it for unit tests, integration tests, and performance measurement with measure blocks.
• Swift Testing - Apple's macro-based library. It runs alongside XCTest, so new unit tests adopt it while UI tests stay on XCUITest.

Cross-Platform Testing Frameworks

Reach for these when one suite has to drive both platforms:

• Appium 3 - The most widely used cross-platform option. Drivers and plugins version independently, so you upgrade one without a full release. More on Appium testing.
• Maestro - Declarative YAML flows with built-in waiting that removes most timing flakiness. Covers Android emulators and real devices, web, React Native and Flutter; on iOS it runs on Simulators only.
• Detox - Built for React Native. It detects app idle state, so tests wait on the app rather than fixed sleeps. It drives real Android devices, but on iOS the Simulator only.
• Flutter integration_test and Patrol - integration_test covers in-app widget flows; Patrol extends it to native permission dialogs Flutter cannot reach.

The same login flow in Appium 3, driving a real Android device through UiAutomator2:

import io.appium.java_client.android.AndroidDriver;
import io.appium.java_client.android.options.UiAutomator2Options;
import org.openqa.selenium.By;
import java.net.URL;

public class LoginTest {

    public static void main(String[] args) throws Exception {

        UiAutomator2Options options = new UiAutomator2Options()
                .setPlatformName("Android")
                .setPlatformVersion("15")
                .setDeviceName("Pixel 9")
                .setApp("/path/to/app.apk");

        AndroidDriver driver = new AndroidDriver(
                new URL("http://127.0.0.1:4723"), options);

        driver.findElement(By.id("com.example.app:id/email")).sendKeys("qa@example.com");
        driver.findElement(By.id("com.example.app:id/password")).sendKeys("secret");
        driver.findElement(By.id("com.example.app:id/login_button")).click();

        String heading = driver.findElement(By.id("com.example.app:id/home_title")).getText();
        System.out.println("Logged in screen: " + heading);

        driver.quit();
    }
}

Point the same script at a cloud grid instead of 127.0.0.1 and it runs across your whole device matrix in parallel rather than one device at a time.

When a test turns flaky, the question worth asking first is whether it ever needed to leave the app process. That one check has fixed more flakiness for me than any retry setting.

Frameworks to avoid starting new projects on

Several frameworks still appear in older tutorials but are no longer maintained. You may inherit them in a legacy suite; do not pick them for new work.

• monkeyrunner - Google's own documentation states that the monkeyrunner API is unmaintained and recommends UI Automator instead.
• Calabash - Microsoft discontinued its contributions, and the calabash-android repository states the project is looking for a maintainer.
• EarlGrey 1.0 - The EarlGrey repository marks 1.0 deprecated in favour of 2.0, unmaintained internally since iOS 13.
• Robotium and Selendroid - Both predate and are superseded by Espresso and Appium 3, which cover the same Android UI cases.

Why Do Mobile Tests Pass Locally but Fail in CI?

Because the CI environment is not your machine. System animations, missing Play Services, no GPU acceleration, a different locale, and slower runners each break assumptions your local device hides.

This is the single most common complaint I hear about mobile suites, and it is almost never the test logic that is wrong. It is an environment difference nobody wrote down.

Work through these causes in order before touching the test code:

On the first row, Android's Espresso setup guide is explicit: it recommends turning off system animations on any device used for testing, naming those exact three settings.

The pattern behind all five is the same. A local device accumulates state and configuration you stopped noticing, and CI starts from nothing every time.

Mobile App Testing Best Practices

Mobile app testing best practices come down to four things: match the device matrix to your real install base, use emulators and simulators only for the checks they can actually prove, automate the paths that have stopped changing, and test the conditions the app runs in rather than the ones it was built in.

1. Address Device Fragmentation

Your device matrix decides coverage, so it has to track the install base rather than the hardware the team happens to own. Two habits keep it honest as the market moves:

• Prioritize platforms - Cover the current and previous iOS versions. On Android, test flagship and budget hardware to expose performance differences.
• Use data - Read usage metrics to find the devices and OS versions your customers actually run, then prioritise those.

2. Use Virtual Devices for the Right Checks

You do not need a physical library of every phone. Split the work by what each environment can actually prove:

• Simulators and emulators - Android Studio and Xcode run virtual devices, fast for checking screen resolutions and OS versions.
• Cloud device labs - Remote access to physical hardware, more reliable than emulation for hardware-specific defects.

3. Implement Automation Strategically

Automation complements manual testing, supplying release speed without losing the human judgment that UX work needs.

• Pick by platform count - Espresso and XCUITest for single-platform apps, Appium 3 or Maestro when one suite must drive both.
• Automate what is stable - Scripts written against a settled UI survive; scripts written against a moving one become maintenance work.
• Run in parallel - Executing the suite across the device matrix at once is what keeps regression time flat as coverage grows.
• Watch Android vitals - Google Play sets Android vitals bad behaviour thresholds of a 1.09% user-perceived crash rate and a 0.47% user-perceived ANR rate, and may reduce the visibility of a title that exceeds them. Crash and ANR checks on low-end devices keep you under that line.

4. Test Environmental Factors

Mobile apps behave differently based on the user's surroundings. Ensure you test:

• Connectivity - Switch between WiFi, 4G, and 3G to see how the app handles low bandwidth or connection drops.
• Permissions - Verify how the app behaves when users deny or grant permissions (camera, location, etc.).
• Hardware interactions - Test multiple taps, gestures, and how the app handles system interruptions like phone calls.

If you are preparing for a role rather than a release, these mobile testing interview questions cover the same ground from the interviewer's side.

What Are the Most Common Mobile App Testing Mistakes?

Testing only flagship devices, trusting emulators for hardware behaviour, testing on fast WiFi alone, skipping the upgrade path, ignoring interruptions, and automating a UI that has not settled yet.

Most escaped bugs are not exotic. They come from habits that look reasonable until something ships broken.

• Testing only on flagship devices - Out-of-memory crashes appear first on low-RAM handsets, which is exactly where nobody is looking.
• Treating emulators as sufficient - A suite green only on emulators has not tested battery drain, thermal throttling, biometrics, or camera behaviour.
• Testing only on fast WiFi - Office broadband hides blocking network calls that freeze the same screen for fifteen seconds on 3G.
• Skipping the upgrade path - Real users upgrade over an existing install, which is where migration bugs and lost sessions appear.
• Ignoring interruptions - State restoration is rarely covered by scripted happy-path tests, so it fails in production instead.
• Automating too early - Scripts written against a changing UI spend their life being repaired. Automate flows once they stabilise.
• Letting flaky tests stay green-ish - A suite that fails randomly gets ignored, and an ignored suite catches nothing. Quarantine and fix.
• Never revisiting the device matrix - A list built two years ago stops matching your install base. Review it quarterly.

The upgrade path is the one I see skipped most often, and it is where the ugliest defects live. A clean install hides every migration bug you have.

Run Mobile App Testing at Scale With Kane CLI by TestMu AI

Start with one flow your team re-tests every release, such as sign-in, and hand it to Kane CLI. Install it with npm install -g @testmuai/kane-cli, describe the flow in plain English, and kane-cli generate drafts the scenarios and test cases.

The CLI is free to install, and local runs are free.

• Plain-English authoring - Describe a flow once and Kane CLI drafts scenarios with step-by-step cases; refine them with --refine instead of rewriting.
• Native app runs without Appium setup - On macOS with Apple Silicon, --target emulator or --target simulator runs the objective against your .apk or iOS .zip build on a local Android emulator or iOS simulator.
• Built for CI and coding agents - The --agent flag emits NDJSON that a pipeline or an AI coding agent can parse for a pass or fail result.
• Real devices for the release gate - Run the release candidate on TestMu AI's 10,000+ real Android and iOS devices, with video, device logs, and network logs from every session.
• Security - TestMu AI is SOC 2 Type II and ISO 27001:2022 certified, with TLS 1.2 or higher in transit and AES-256 at rest, which matters because a signed .apk or .ipa is production code leaving your perimeter.

Setup steps are in the Kane CLI installation guide, and the app testing on real devices documentation covers the device cloud side.

Real device testing starts with five free sessions of two minutes each a month, and paid plans are on the TestMu AI pricing page.

For hands-on sessions as well as automated runs on the same grid, the mobile app testing platform streams 10,000+ real Android and iOS devices into the browser.