Smoke Testing: Definition, Examples & Checklist [2026]

Smoke testing is a shallow, broad set of tests run against a fresh build to confirm its critical functions work before deeper testing begins.

A typical smoke suite holds 20 to 50 test cases and finishes in 5 to 15 minutes. If it fails, the build is rejected and returned to development rather than entering regression.

It is also called build verification testing, build acceptance testing, or confidence testing. All four names describe the same gate: a cheap check that answers one question before anyone spends real time on a build.

That question is not "does this build work?" but "is this build worth testing?".

The distinction matters. It explains why a smoke suite deliberately skips edge cases, invalid inputs, and validation logic that a thorough test plan would insist on.

Cost is the argument that settles it. A regression suite runs for hours, so finding a broken login page afterwards wastes a full cycle of QA capacity that a 10-minute gate would have protected.

By the end of this guide you will have a 12-case smoke suite, a pipeline that blocks promotion when it fails, and four metrics that tell you whether the gate still works.

What Is Smoke Testing

A smoke test checks that the handful of journeys a product cannot ship without still complete end to end. It proves a build is not obviously broken, rather than proving it is correct.

Put concretely, it verifies the handful of functions a product cannot ship without are still working after a change.

On a banking app that means logging in, viewing a balance, and making a transfer. A video platform checks signing in, loading a video, and pressing play.

Each check is binary and shallow. The test confirms the path completes, and nothing more: it does not verify that the transfer amount rounds correctly, or that the video buffers at the right bitrate.

The car analogy is the one most testers learn first. Before a new vehicle leaves the factory, someone checks that it starts, that the engine runs, and that it moves and stops.

Nobody measures paint depth at that stage, because a car that will not start makes every finer measurement irrelevant.

Software works the same way. Smoke tests run immediately after a build is produced and again before release, and they exist to make the expensive testing that follows worth doing.

A software build converts source code into a stand-alone form that can run on any system.

That conversion carries risk. Configuration drift, regression, and environment differences can all leave a build that compiles but does not run where you need it.

So the first build goes through smoke testing before reaching any other testing level. If key features are broken, there is no reason to invest time in deeper testing.

Smoke testing, also called build verification or user acceptance testing, is a preliminary software analysis that checks the functionality of important parts of a program but does not determine whether the program is error-free.

Why is it called smoke testing?

The name has two origins, and both describe the same idea. In plumbing, smoke is forced through a closed system of pipes so that any crack or leak reveals itself immediately, without dismantling anything.

In electronics, a newly assembled board passed its smoke test if it did not emit smoke the first time power was applied.

Neither check proves the system is correct. Both prove it is not obviously, catastrophically broken, which is precisely the question a software smoke suite answers.

The software practice was formalised by Steve McConnell in IEEE Software, July 1996, documenting Microsoft's daily build and smoke test routine.

When to Use Smoke Testing

Run smoke tests at the start of any testing cycle, immediately after a new build reaches a test environment, and again after any code or infrastructure change that affects a critical path.

Smoke tests are build verification tests, generally used at the start of any testing cycle for new builds.

They confirm the primary requirements are met and stop the build from being deployed when they are not. Run them as soon as a new build is deployed, or after any change.

Code needs to be stable before regression testing can begin, so smoke tests are created in each sprint to ensure code stability. This saves time and ensures that the corners of the code are covered.

Smoke testing aims to test a system's most important functionalities, not to execute all possible test scenarios.

In practice, there are five moments where a smoke test earns its place in the workflow:

• After every new build is deployed to a test environment. The classic build-acceptance gate: QA either accepts the build or returns it.
• Before a pull request is merged. A reduced suite on the feature branch stops broken changes reaching main.
• Before the regression suite starts. Regression runs take hours; spending them on a broken login page wastes QA capacity.
• Immediately after a production deployment. Catches environment-specific failures, such as a missing config value, before users find them.
• After a third-party or infrastructure change. Gateway swaps, DNS changes, and dependency upgrades break critical paths without touching your code.

The common thread is that each moment carries a cheap check and an expensive consequence. Smoke testing is worth running wherever a few minutes of verification protects hours of downstream work.

Why Is Smoke Testing Important

Smoke testing matters because it produces a single decision cheaply: promote this build, or send it back. That decision protects the hours of regression and exploratory work queued behind it.

Its primary purpose is not to find bugs, but to determine whether the software has reached a workable degree of stability.

Any smoke test assembles checks that are broad in scope rather than deep in defect-hunting, because its output is a single decision about the build.

• Smoke tests flag breakage. Automated and standardised across builds, a failure usually means a wrong file or something basic is broken.
• Pass: the build is promoted to the next stage and QA begins deeper testing against it.
• Fail: the build is rejected and returned to development, and no further QA time is spent on it.
• The outcome is binary by design. A smoke run yields no severity ranking, no defect count, and no partial credit.
• That binary quality is why suite trustworthiness matters. One flaky case turns a clear decision into an argument.
• Until the gate passes, automated functional testing cannot start and the build does not progress.
• Smoke testing also runs during other stages, such as system testing, making build validation faster across every testing type.

What Are the Characteristics of a Smoke Test

A smoke test is fast, broad, shallow, scripted, and repeatable. It covers positive paths only, runs equally well for developers or QA, and stays small enough that its runtime never drifts.

Eight properties define the shape of a smoke test: fast, broad, shallow, scripted, repeatable, positive-path only, runnable by developers or QA, and small enough to stay that way.

• This automated test runs quickly, enabling self-scoring of the test.
• The test also provides broad coverage across the system.
• A test can be run by developers and also as part of the quality assurance process.
• Shows basic errors in a new build, though it doesn't need to be exhaustive.
• An integral part of the development cycle, verifying the build by checking its key features.
• Smoke tests can be performed manually or automatically, depending on the needs of the testing team.
• It applies to different software testing types, including integration, system, and acceptance testing.
• A limited form of testing with few cases, run through positive scenarios with valid data, and fully documented.

What Are the Benefits of Smoke Testing

Smoke testing returns stable builds, earlier defect discovery, protected QA capacity, and lower integration risk. Each one traces back to catching structural breakage before others build on it.

The returns below are specific and measurable, not a general claim to quality.

Below are the advantages that come from performing these checks early in the development process:

• System stability

It's valuable to be able to verify the stability of your builds early on. This allows you to use them later, reducing developers' work manually searching for and reporting bugs late into the development cycle.

Verifying core functions up front means testers can work without second-guessing whether the build itself is at fault.

The positive results of smoke testing are often enough to guarantee system stability, a solid foundation you can build on later.

It is easier to add to something that works than to overhaul a broken system later.

• Simple process

Smoke tests confirm that every critical component responds, and the run itself is inexpensive to set up.

That means you can smoke tests often without overtaxing the resources of your servers and other equipment.

Smoke tests are simple enough to combine with just about any type of testing.

You can add one after nonfunctional testing, run it more than once during development, or place it at the end of hardware testing.

• Identifies bugs quickly

The fewer bugs that reach end-users, the better. Catching them early gives developers time to fix the most prominent ones before they ever ship.

By frequently smoke testing your software, you can keep it relatively free of bugs. You can promise your users that any potentially harmful errors have been caught before they impact their experience.

• Improves end-product quality

Smoke testing leads to fewer bugs. This means a better product for your customers.

When you provide a high-quality product, your customers will be more satisfied, leading to a better reputation for your organization.

Smoke testing is a simple way to catch big bugs early in the testing process. All you need to do is run your application frequently and use it well.

• It makes QA teams’ work more accessible

QA teams have many responsibilities and cannot afford to waste time executing large test suites to catch issues smoke tests could catch.

To maximize the efficiency of your QA testing, running more smoke tests with greater frequency is good. That's because smoke tests help you catch bugs before your QA teams find them.

Smoke testing is an efficient way to ensure QA teams have the time and resources to search for bugs thoroughly. That headroom is what lets you ship on schedule.

• Improves efficiency

A proper build should be free of errors and fully functional, which is why smoke tests are worth relying on: they flag every error affecting stability.

Developers can then remove those errors early, so each subsequent build rests on a solid foundation.

When you smoke test your foundational code, you increase the reliability of your builds. You also integrate new builds more easily because they will be relatively clean of significant issues.

Moreover, smoke testing can improve the efficiency of other test stages. Because those stages won’t be affected by the problems caught by smoke tests, they can be fixed before they affect other stages.

• Lessens integration risks

When we say “integrations” here, we’re not talking about the things that let you stack your tech into one unified app.

Smoke testing helps to identify problems caused by integrating new pieces of code with existing ones. Those problems are typically found by means of fresh builds, each of which would need to be tested first.

When smoke testing is performed on an integrated software system, you can be sure that each component has been validated already. That makes the integration process much smoother and more stable.

Also, you can reduce the chances that your original build will reject the new content by providing proof that the new content is stable and free of software-breaking bugs.

Thanks to smoke testing, you can create apps that easily incorporate new content in your app. This means you can continuously improve the experience you provide for your customers.

• Saves time and resources

Smoke tests can help you detect problems quickly, so your teams won’t have to spend as much time searching through the code. That saves time and resources, especially if you do not use much automation.

As you can see, smoke testing is fully automated if you choose that option. The next point in this list will elaborate on that point.

• You can run automated tests

Automation testing saves time and resources, because automated runs complete far faster than a human working through the same checks by hand.

For a hands-on example, see how to run automated cross-browser smoke checks from n8n.

Automation has many advantages.

• Automating a process eliminates the unpredictability associated with human error
• You can also run tests more often because automation frees your team’s time to focus on critical tasks
• Suite size stops being a runtime constraint once you use parallel testing
• Highly flexible

You do not need automation to benefit from smoke testing. The process is flexible, and the different types covered below mean there is one that fits your team.

Automation suits most teams, but not all. If you would rather not use it, manual smoke testing still works and carries its own advantages.

Smoke testing isn’t just for automated and manual builds; it works on all builds, with minimal adjustments to the actual process.

Any business running software can apply the process. The agility of this process makes it usable by all kinds of businesses.

• Delivers feedback quickly

An automated smoke suite returns a verdict within minutes of a commit, while the developer still has the change loaded in their head.

That is the difference between a two-minute fix and a two-hour archaeology session a week later.

Smoke tests highlight the big, structural breakages. Later stages such as regression and exploratory testing surface the subtler problems once the build has proven it is worth that attention.

• Ensures your API testing process is smooth.

API testing is an important part of the development cycle and can benefit from incorporating smoke testing.

An application programming interface is checked for its expected functions. It also verifies the build handles required security protections consistently, and assesses overall reliability.

API penetration testing tools strengthen the security posture of the service under test.

Smoke testing catches bugs first, and your API testers won't have to waste time looking for things that aren't there. That's how smoke testing helps you ensure your API testing is effective.

What Are the Limitations of Smoke Testing

A smoke suite cannot find edge-case defects, judge correctness, or replace regression. Every limitation follows from the shallowness that makes the suite fast enough to gate builds.

Despite its many benefits, smoke testing has a few disadvantages that you should be aware of. There is no one-size-fits-all solution. Here are some of the disadvantages mentioned below:

• Smoke testing requires documentation to be correctly done, so a specialized testing team is required.
• These tests are not meant to replace full-scale functional testing.
• When smoke testing a software build, it can sometimes be a waste of time if the build is not stable.
• For minor application changes, a full smoke test across the whole application is not worth running.
• The tests will not be run against negative test cases or invalid input.
• Even after testing the whole application for bugs, you may find critical issues arise in integration and system testing.
• Any bug left in smoke testing causes larger problems later, so eliminate them before the next phase.
• Any software project requires time and money, and smoke testing is fully scripted, so special manpower is needed.

Smoke Testing vs Sanity Testing

Smoke testing and sanity testing are two different approaches, though both aim at software quality, lower integration risk, and saved time.

The table below lists the main differences between them.

For a fuller side-by-side treatment with worked examples of each, read our dedicated guide to smoke testing vs sanity testing.

What Is the Difference Between Smoke Testing and Regression Testing

Smoke testing runs 20 to 50 shallow checks in minutes to decide whether a build is worth testing. Regression testing runs deep suites for hours to confirm nothing existing has broken.

These two are often confused because both run against a new build, but they answer opposite questions.

The first asks whether the build is worth testing at all. The second asks whether anything that used to work has stopped working.

In practice the relationship is sequential: smoke is the gate, and regression is what runs once the gate opens.

Running regression on a build that would have failed a smoke test is the most common way QA capacity gets burned.

The two suites also age differently. A smoke suite should stay roughly the same size forever, because the number of critical paths changes slowly.

A regression testing suite grows with every feature and every bug fix, which is exactly why it cannot double as a gate.

How Does Smoke Testing Work

Smoke testing works as a gate. The development team hands over a build, QA runs a small suite of critical-path checks against it, and the result decides whether deeper testing begins.

Every smoke test run follows five steps, from build handover to environment cleanup:

• After a build is deployed by the development team, it is sent to the testing team for testing.
• After the QA team receives the builds, they design test cases based on the requirements.
• A smoke test suite contains many tests collected together into a single package for efficiency and convenience.
• Automating the smoke test can be efficient and cost-effective, so consider automating it if you want to save time.
• Finally, execute the test cases and clean the environment for the next round: stop servers, delete files, empty database tables.

What Does a Smoke Test Case Look Like

A smoke test case is deliberately thin. It names one critical path, one action, and one binary expected result, with no boundary values, invalid inputs, or error-message assertions.

Each case is deliberately thin, naming one critical path, one action, and one binary expected result.

There are no boundary values, no invalid inputs, and no error-message assertions, because every one of those belongs in the regression suite.

Below is a complete 12-case smoke suite for an e-commerce application. It covers every revenue-critical path and runs in roughly eight minutes when automated.

Notice what is absent. There is no test for an invalid password, no check that the cart rejects a quantity of zero, and no verification of discount-code arithmetic.

Those are all valuable tests, and all of them belong in regression.

The bloat never arrives in one commit. It comes one reasonable-sounding case at a time, which is why I treat any new smoke case as a regression case until proven otherwise.

Adding them to the smoke suite is the most common way teams turn a 5-minute gate into a 40-minute one, at which point developers start skipping it.

The same test case as automation code

Here is ST-02 through ST-06 written as a single Playwright smoke spec. The whole file is short by design, which is what makes it cheap to run on every commit.

const { test, expect } = require('@playwright/test');

test.describe('@smoke critical paths', () => {
  test('user can log in', async ({ page }) => {
    await page.goto('/login');
    await page.fill('#email', process.env.SMOKE_USER);
    await page.fill('#password', process.env.SMOKE_PASS);
    await page.click('button[type="submit"]');
    await expect(page.locator('[data-test="dashboard"]')).toBeVisible();
  });

  test('user can search and open a product', async ({ page }) => {
    await page.goto('/');
    await page.fill('[data-test="search"]', 'wireless keyboard');
    await page.press('[data-test="search"]', 'Enter');
    await expect(page.locator('[data-test="result-card"]').first()).toBeVisible();

    await page.locator('[data-test="result-card"]').first().click();
    await expect(page.locator('[data-test="add-to-cart"]')).toBeEnabled();
  });

  test('user can add a product to the cart', async ({ page }) => {
    await page.goto('/product/wireless-keyboard');
    await page.click('[data-test="add-to-cart"]');
    await expect(page.locator('[data-test="cart-count"]')).toHaveText('1');
  });
});

Tagging the block with @smoke is the detail that matters most.

It lets one repository hold a single test suite while the pipeline selects only the smoke subset at the gate, and the full set later.

How Do You Run Smoke Tests in a CI/CD Pipeline

Run the smoke suite as an automated pipeline stage triggered once the build deploys to a test environment, then make the regression stage depend on it so any failure blocks promotion.

I would rather see a smoke suite fail loudly and often than one that retries its way to green, because the second kind quietly teaches a team to stop reading the results.

In a modern delivery pipeline, smoke testing is not a phase that someone schedules. It is a gate that runs automatically on every build and blocks promotion when it fails.

The stage sits in a fixed position: after the build artifact is deployed to a test environment, and before the regression stage is allowed to start. Three rules keep it useful.

• Fail fast. Abort on the first failure rather than completing the suite. You already have your answer, and delay costs fix time.
• Block promotion, not just report. A stage that logs a warning and continues is no gate. Regression must depend on the result.
• Never retry a smoke failure automatically. Retries hide breakage. A test flaky enough to need them is itself the defect.

GitHub Actions smoke gate

This workflow deploys the build, runs only the @smoke tagged tests, and makes the regression job depend on the smoke job succeeding. If smoke fails, regression never starts.

name: build-and-verify

on:
  push:
    branches: [main]

jobs:
  smoke:
    name: Smoke gate
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npm run deploy:staging

      - name: Run smoke suite
        run: npx playwright test --grep @smoke --max-failures=1
        env:
          SMOKE_USER: ${{ secrets.SMOKE_USER }}
          SMOKE_PASS: ${{ secrets.SMOKE_PASS }}

  regression:
    name: Regression suite
    needs: smoke          # gate: runs only if smoke passed
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx playwright test --grep-invert @smoke

The two lines doing the real work are needs: smoke and --max-failures=1.

Together they turn a reporting stage into an enforcing one: the first creates the dependency, and the second stops the run the moment the build is proven broken.

A passing run prints the suite summary and lets the regression job start:

Running 12 tests using 4 workers
  12 passed (7.4s)

Smoke gate .................. success
Regression suite ............ queued

When a case breaks, the run stops there and the regression job never leaves the queue:

  1 failed
    [Chrome] @smoke critical paths > user can log in
    Error: expect(locator).toBeVisible() failed

Smoke gate .................. failure
Regression suite ............ skipped

The timeout-minutes: 15 ceiling is deliberate too. It turns the runtime budget into something the pipeline enforces, so a slowly bloating smoke suite fails loudly instead of quietly becoming a bottleneck.

Running the smoke gate across browsers in parallel

A smoke suite that only runs on headless Chrome will miss browser-specific breakage until regression, which defeats the point of gating early.

Pointing the same suite at a cloud grid keeps runtime flat while widening coverage, because the cases execute concurrently rather than in sequence.

// playwright.config.js - fan the @smoke suite across browsers
const browsers = [
  { browserName: 'Chrome',        browserVersion: 'latest', platform: 'Windows 11' },
  { browserName: 'MicrosoftEdge', browserVersion: 'latest', platform: 'Windows 11' },
  { browserName: 'pw-webkit',     browserVersion: 'latest', platform: 'macOS Sonoma' },
];

const wsEndpoint = (b) => {
  const capabilities = {
    browserName: b.browserName,
    browserVersion: b.browserVersion,
    'LT:Options': {
      platform: b.platform,
      build: 'Smoke gate',
      user: process.env.LT_USERNAME,
      accessKey: process.env.LT_ACCESS_KEY,
    },
  };
  return `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`;
};

module.exports = {
  grep: /@smoke/,
  workers: browsers.length,   // every browser runs at once
  retries: 0,                 // never retry a smoke failure
  projects: browsers.map((b) => ({
    name: `${b.browserName}-${b.platform}`,
    use: { connectOptions: { wsEndpoint: wsEndpoint(b) } },
  })),
};

Because the browsers run concurrently, testing three of them costs roughly the same wall-clock time as testing one.

You can run the same suite on an online browser farm of 10,000+ browser and OS combinations without moving past the 15-minute budget.

What Are the Types of Smoke Testing

Smoke testing splits into formal and informal, divided by who authorises the run. Formal runs are requested by a test lead and reported back; informal runs start on a tester judgement.

Independent of how you execute it, smoke testing splits into two types by who authorises the run: formal and informal

Formal smoke testing

Here the development team sends the application to the test lead, who instructs the testing team to run smoke testing.

Once that team completes the run, they send the smoke testing report back to the test lead.

Informal smoke testing

The Test lead says that the application is ready for further testing. The test leads do not specify to do smoke testing, but still, the testing team starts testing the application by doing smoke testing

Should You Run Smoke Tests Manually or Automate Them

Execution splits three ways: manual, automated, or hybrid. Build frequency decides it, because a manual run costing 20 minutes of a tester per build stops scaling once you deploy more than once a day.

Smoke testing usually is performed manually, but it may also be accomplished through automation. It differs from organization to organization

Manual Testing

To test a product, the tester writes and updates the test cases, covering either existing features or new ones.

Manual smoke testing is the most common method, applied to the fresh build and any newly added features. Scripts must be modified to reflect each requirement.

Smoke testing ensures critical path navigation does not hinder functionality. Once the build reaches QA, high-priority functional test cases run to surface critical defects.

If those pass, functional testing continues. If they fail, the build is rejected and returned to the development team for correction.

QA then receives a new build and begins with smoke testing again.

The run covers new builds integrated with older ones to confirm system correctness. Before starting, QA should verify each build carries the correct version.

Automation Testing

Automation testing tools handle the testing process automatically. Automated smoke tests are performed when we need to run a batch of automated tests and immediately test the build whenever an issue is reported

Smoke testing can drastically reduce test time. Automated, it can finish in only a few minutes.

Where developers build frequently or run continuous testing strategy, automating smoke tests lets QA give faster feedback on each build.

Tools for automated smoke testing include TestMu AI and PhantomJS.

TestMu AI is a cloud-based cross browser testing platform that runs automation at scale across an online browser farm of 10,000+ browsers and operating systems.

You can subscribe to the TestMu AI YouTube Channel and stay updated with the latest tutorials around automation testing, Cypress UI testing, Responsive testing, and more

Hybrid Method

The hybrid method combines manual and automated testing.

Testers write the test cases and can also automate them with tools, which raises performance by drawing on both approaches.

As its name suggests, hybrid smoke testing combines functional and performance testing basics methodologies. Its purpose is to improve the overall effectiveness of smoke testing

Which Tools Are Used for Smoke Testing

There is no dedicated smoke testing tool. A smoke suite combines an authoring framework such as Playwright or Selenium, a CI server such as Jenkins, and a parallel cloud execution grid.

No tool on the market is built solely for smoke testing.

A smoke suite is built from the same components as any other automated suite, so the real decision is which authoring framework, CI server, and execution grid fit your stack.

The table below maps the common choices to the job each one actually does in a smoke gate.

One selection rule matters more than the individual choices: whatever you pick must support tagging or grouping.

A smoke suite that lives in a separate repository from the regression suite will drift out of sync within a few sprints.

If you are still choosing a framework, our comparison of automation testing tools covers the trade-offs in more depth.

Which Metrics Should You Track for Smoke Testing

Track four numbers: build acceptance rate, smoke suite runtime, escaped defect rate, and smoke flakiness. Together they show whether the gate is still doing the job it was built for.

Most teams track smoke results as a simple pass or fail and stop there, which throws away the signal.

A smoke gate is a sensor on build health, and four numbers tell you whether it is doing its job.

Escaped defect rate is the one most worth watching.

If I had to keep one of these four, it would be escaped defect rate, because it alone tells you what the suite is missing rather than how it ran.

Every critical-path bug that reaches regression is direct evidence that a case is missing from the smoke suite, which makes it the cleanest input for what to add next.

Flakiness deserves the harshest treatment. A smoke gate only works if a red result is believed, and one unreliable test is enough to teach a team to re-run instead of investigate.

How to Perform Smoke Testing on TestMu AI

On TestMu AI, define the smoke suite in a hyperexecute.yaml file, run it as a single pipeline step, then use a real-time session to reproduce any failure by hand on a live browser.

Now that you know what belongs in a smoke suite and where the gate sits in the pipeline, here is how to run it on TestMu AI.

Start with the automated path, since that is what a smoke gate needs.

The manual real-time sessions that follow suit the exploratory check a tester runs when a smoke test fails and someone must see the breakage first-hand.

Running the smoke gate on HyperExecute

HyperExecute is the execution layer for the gate itself.

It shards the suite across parallel runners so a 12-case smoke suite finishes in minutes, and returns a single pass or fail exit code your pipeline can gate on.

Define the gate in a hyperexecute.yaml at the repository root:

version: 0.1
runson: linux
autosplit: true
concurrency: 4          # four runners share the smoke suite
maxRetries: 0           # never retry a smoke failure

testDiscovery:
  type: raw
  mode: dynamic
  command: npx playwright test --grep @smoke --list

testRunnerCommand: npx playwright test --grep @smoke $test

report: true
partialReports:
  location: playwright-report
  type: html

Then invoke it as a single pipeline step. The CLI exits non-zero when any smoke case fails, which is what lets the regression stage depend on it:

./hyperexecute --config hyperexecute.yaml \
  --user $LT_USERNAME --key $LT_ACCESS_KEY

The two settings that make this a gate rather than a test run are maxRetries: 0 and autosplit: true.

One preserves the signal by refusing to mask flaky failures. The other keeps runtime inside the budget as the suite grows.

Full option reference: deep dive into HyperExecute YAML.

Real Time Website and Web App Testing

When a smoke test fails and you need to reproduce the breakage by hand, a real-time session is the fastest way to see it. Here's how to run one on the TestMu AI platform:

• Register and log in to your TestMu AI account.
• Click the Real Time Testing tab in the left menu bar.

• Enter the URL in the Real Time Testing window and choose the BRAND, DEVICE/OS, and BROWSER.

• Click START. You can launch your website by using the required configuration.
• Once launched, you can test the site using screenshots, screen recording, one-click bug logging, and more.
• Test responsiveness with the LT Browser tool, which offers hot reloading, network throttling, and side-by-side device interaction.

Web Automation and App Testing

To perform automated smoke testing on the TestMu AI platform, follow the below steps:

• To automate smoke tests, go to the Automation section of the left menu bar.

• You can run automated tests on the TestMu AI platform, then pick your preferred framework or language on screen.
• Add OS and browser information, specify required capabilities, set the project, then select your options and test.


You can also use the TestMu AI platform to perform mobile app testing for smoke testing on Emulators/Simulators or a real device cloud.

Real Time Mobile App Testing

On TestMu AI, you have two options for mobile app testing:

• Testing with emulators and simulators
• Testing with real devices

TestMu AI offers both options, and it is up to you which way to go.

To smoke test with emulators or simulators on the TestMu AI platform, please follow these steps:

• Click the Real Time Testing option in the left menu bar to smoke test your mobile application.
• You can upload the app and select the device or OS that needs to be tested from the App Testing tab.

• Press the START button to begin the test session.

For Real Devices on the TestMu AI platform, follow these steps:

• Go to Real Device under the left menu, and choose Real Time or App Automation.
• Real Time on a device focuses on manual testing. You can also run app automation with Appium, Espresso, or XCUITest.

Now you can interact with the app.

TestMu AI allows you to perform Real Device Cloudon an online device farm. Watch this video to learn more about how to test your applications on TestMu AI:

What Are the Best Practices for Smoke Testing

Run the suite early and unconditionally, keep it capped and fast, track results per build, and remove flaky cases on sight. Each practice protects the property that makes the gate worth having.

Seven practices separate a smoke suite that holds its value from one that decays into a formality.

• Run tests often and early: Catch errors before the build becomes unstable, which is why early runs pay off.
• Never skip any testing stage: Assuming a build is stable is tempting, but validity is unknowable without testing it.
• Test everything: Test every build, whether it moves into integration or performance testing. Smoke testing suits every build type.
• Keep an eye on the checklist: A website testing checklist helps manual testers, and anyone organising a testing process.
• Track your test results: A build that keeps failing needs addressing, and one that always passes is worth knowing about too.
• Test at least two times: Running builds through smoke testing more than once raises your chances of stopping major bugs.
• Choose the right testing type: On a limited budget, a manual testing approach may not be feasible; hybrid or automated fits better.

The smoke testing checklist

Run through this before you accept a smoke suite as your build gate. If you cannot tick every line, the suite will either miss breakage or slow the pipeline down.

SMOKE TESTING CHECKLIST

Scope
[ ] Every revenue-critical or rollback-worthy user path has exactly one case
[ ] No negative cases, boundary values, or invalid inputs in the suite
[ ] No field-level or error-message assertions
[ ] Suite size is between 20 and 50 cases

Speed
[ ] Full suite completes in under 15 minutes
[ ] Suite runs in parallel, not sequentially
[ ] Pipeline enforces a hard timeout on the stage

Pipeline
[ ] Triggered automatically on every deploy to the test environment
[ ] Runs before the regression stage, never after
[ ] A failure blocks promotion rather than logging a warning
[ ] Configured to fail fast on the first failure
[ ] Automatic retries are disabled

Reliability
[ ] Test data is seeded, not dependent on prior runs
[ ] Test accounts and secrets come from the CI secret store
[ ] Environment is reset or torn down after each run
[ ] Flaky tests are removed from the suite until fixed

Coverage across environments
[ ] Suite runs on every browser and OS your users actually use
[ ] A reduced suite runs against production after each deployment
[ ] API-level critical paths are covered, not just the UI

Measurement
[ ] Build acceptance rate is tracked over time
[ ] Suite runtime trend is tracked, not just the latest value
[ ] Every critical-path defect escaping to regression triggers a new smoke case

The last line is the one that keeps the suite honest over time. Treating each escaped defect as a missing smoke case is what stops the gate from slowly decaying into a formality.