How to Fix Flaky Tests?

To fix a flaky test, confirm it is actually flaky by rerunning it several times, reproduce it in isolation, and capture logs, screenshots, video, and traces from the failures. Then identify the root cause, usually timing and waits, shared state, test data, the environment, or an external dependency, and apply the matching fix such as smart waits instead of sleeps, stable selectors, mocked dependencies, and isolated state. Verify the fix holds across repeated runs, and quarantine plus track any test you cannot fix immediately so it never silently blocks your pipeline.

This guide is about repairing tests that are already flaky. To stop new ones from appearing in the first place, see How to Avoid Flaky Tests?

The Flaky Test Repair Loop at a Glance

Fixing flakiness is a repeatable loop, not a guess. Work through these steps in order:

• Confirm: rerun the test many times to prove it really is flaky and not a genuine regression.
• Isolate: reproduce the failure on its own, then under varied order and parallelism.
• Capture: collect logs, screenshots, video, network logs, and traces from the failing runs.
• Diagnose: map the failure to a root cause from the taxonomy below.
• Fix: apply the specific repair for that cause.
• Verify: re-run the repaired test repeatedly until it is consistently green.
• Quarantine and track: for anything you cannot fix now, isolate it so it runs but does not block the pipeline.

What Makes a Test Flaky?

A flaky test is one that produces different results, passing on one run and failing on the next, against the exact same code and inputs. The outcome is non-deterministic, so a red build no longer reliably means a real defect. This erodes trust fast: industry surveys report that a majority of developers deal with flaky tests on a monthly, weekly, or even daily basis, and Google's testing research found that a large share of pass-to-fail transitions in their continuous integration were flaky failures rather than true regressions. Once teams stop trusting the suite, they start ignoring failures, and that is when real bugs slip through.

Step 1 - Confirm the Test Is Actually Flaky

Before you change anything, prove the test is non-deterministic rather than consistently broken. Run it repeatedly and observe whether the result flips:

• Rerun in a loop: execute the test 20 to 50 times locally without changing any code.
• Rerun in CI: repeat the loop in your pipeline, where timing, load, and resources differ from your laptop.
• Classify the result: if it passes sometimes and fails sometimes, it is flaky. If it fails every time, it is a genuine regression, fix the underlying defect instead.
• Handle non-reproducers: if it never fails across dozens of runs, log it, note the conditions, and deprioritize rather than chasing a ghost.

Step 2 - Reproduce and Isolate the Failure

A test that flakes only inside the full suite is usually being affected by something outside itself. Narrow it down:

• Run it alone: execute only this test. If it now passes consistently, another test is polluting shared state.
• Vary the order: randomize or reverse test order to expose hidden dependencies between tests.
• Run in parallel: execute the suite concurrently to surface race conditions and shared-resource contention.
• Inspect the predecessor: look at the test that runs immediately before it, leaked sessions, cookies, or database rows are a common culprit.

If the failure only shows up in the browser layer, step through it with the techniques in How Do I Troubleshoot and Debug Selenium Tests?

Step 3 - Capture Failure Evidence

Flaky failures are hard to fix precisely because they are rare, so capture everything the moment they happen:

• Logs and console output: capture application logs, browser console errors, and the full test stack trace.
• Screenshots and video: record the screen at the point of failure to see exactly what state the UI was in.
• Network logs: save request and response timing to spot slow or failed calls to back-end services.
• Framework traces: enable Selenium or Playwright traces so you can replay the run step by step.
• Historical run data: keep the pass and fail history across builds so you can correlate failures with specific environments or times.

Running tests on a cloud grid such as TestMu AI Selenium Automation helps here, because each session automatically records video, console, and network logs, giving you a captured artifact for every flaky run instead of an unreproducible one-off.

Step 4 - Identify the Root Cause

Almost all flakiness falls into a handful of categories. Match the evidence from Step 3 to one of these root causes, then jump to the matching fix in Step 5.

Step 5 - Apply the Fix for Each Root Cause

This is where the test actually gets repaired. Apply the fix that matches the cause you diagnosed:

• Timing and async: replace hardcoded sleeps and fixed timeouts with explicit or smart waits that poll for a specific condition, the element being clickable, a network call completing, or a spinner disappearing.
• Brittle selectors: switch to stable, unique locators such as a dedicated data-testid, and avoid positional XPath or index-based selectors that break on minor markup changes.
• Test data: use fixed or mocked data through factories and fixtures, generate isolated records per test, and never depend on shared or production data that other tests can mutate.
• Shared state and order: reset state before every test, isolate each test fully, and set up preconditions through the API rather than driving the UI, for example authenticating with a token instead of the login form.
• External services: stub or mock third-party APIs so a slow or unavailable dependency cannot fail your test, and seed any randomness so behaviour is repeatable.
• Environment: run on clean, consistent, isolated environments so machine-specific state never leaks between runs. If your tests time out under load, the patterns in Why Are Selenium Tests Running Slow or Timing Out? apply directly, and Playwright users can follow Why Are Playwright Tests Running Slow or Timing Out?

When the flakiness traces back to test data living in spreadsheets or external files, externalizing and stabilizing that data, as covered in What Is Apache POI in Selenium?, removes another common source of non-determinism.

Step 6 - Verify the Fix Actually Holds

A single green run does not prove a flaky test is fixed, the original failure was rare to begin with. Re-run the repaired test under the same pressure that exposed it:

• Re-run repeatedly: execute the fixed test 50 or more times before you consider it stable.
• Re-run under stress: run it in parallel and in varied order, the same conditions that first surfaced the flake.
• Run in CI, not just locally: confirm it stays green in the pipeline environment as well.
• Only then close it: mark the test as fixed when it is consistently green across all of the above, never on the first pass alone.

Step 7 - When You Can't Fix It Now, Quarantine and Track

Sometimes you cannot root-cause a flaky test immediately, but you also cannot let it block every build. Quarantine it responsibly:

• Quarantine, don't delete: move the test to a quarantine group so it still runs and reports but does not fail the pipeline.
• Tag and track it: record it in your test management system as test debt with an owner and a ticket, not an indefinite "ignore."
• Set a flake-rate threshold: a healthy suite sits around a one to two percent flake rate; treat five percent or higher as a problem that needs action.
• Keep retries minimal: if you enable retries, keep them low and temporary. Retries are mitigation that hide flakiness, they are not a fix, and a test that only passes on retry is still broken.

Detecting and Fixing Flaky Tests at Scale

Manually rerunning tests works for one flaky case, but across thousands of tests and builds you need data. TestMu AI mines historical run data to flag tests that flip non-deterministically, ranks them by severity based on flake rate, and groups failures by likely root cause, so you can target the worst offenders first instead of triaging blindly. You can read more in the Analytics Modules Test Intelligence Flaky Test Analytics documentation.

The cloud grid also makes reproduction far easier. Running the suspect test in parallel across real browsers and operating systems reproduces environment and timing flakiness quickly, and every session ships with the video, console, and network artifacts you need for Step 3, turning a "can't reproduce" failure into one you can actually debug.

Stop Them Coming Back

Repairing a flaky test is only half the job, the same patterns will reappear if your tests are not designed for stability. Bake in stable selectors, explicit waits, isolation, and mocked dependencies from the start so new flakiness never gets a foothold. For the full proactive playbook, see How to Avoid Flaky Tests?

Frequently Asked Questions

What causes flaky tests?

Most flakiness comes from timing and async waits, race conditions, test order dependency and shared state, unstable environments, third-party or external service dependencies, non-determinism such as system time and random values, and brittle element selectors. Diagnosing which one applies is the key to a lasting fix.

How do I detect flaky tests?

Rerun the test many times under identical conditions and watch for inconsistent results, then track pass and fail history across builds in CI. Test analytics that mine historical run data can surface tests that flip non-deterministically across builds and rank them by flake rate so you do not have to find them by hand.

Should I use retries to fix flaky tests?

No. Retries hide flakiness, they do not fix it. Use minimal, temporary retries only as mitigation to keep the pipeline moving while you root-cause and repair the test. A test that only passes on a retry is still broken and still erodes trust in your suite.

What is quarantining a flaky test?

Quarantining isolates a flaky test so it still runs and reports its result but does not block the pipeline or fail the build. You tag it, track it as test debt with an owner, and remove it from quarantine once it is fixed and verified, so it is a holding pattern, not a place to forget tests forever.

How many times should I rerun a test to confirm it's flaky?

Typically 20 to 50 runs, ideally both locally and in CI where conditions differ. If it fails inconsistently across those runs it is flaky. If it never reproduces, it is likely not flaky, or too rare to chase right now, so log the conditions and deprioritize rather than burning time on it.

How is fixing flaky tests different from avoiding them?

Fixing is reactive: you repair tests that already flake by diagnosing and correcting the root cause. Avoiding is proactive: you design tests so they never flake, using stable selectors, explicit waits, isolation, and mocked dependencies from the start. See How to Avoid Flaky Tests? for the prevention side.

• How to Avoid Flaky Tests?
• How Do I Troubleshoot and Debug Selenium Tests?
• Why Are Selenium Tests Running Slow or Timing Out?
• Why Are Playwright Tests Running Slow or Timing Out?
• What Is Apache POI in Selenium?