Release Testing: Release Candidates and Exit Criteria

It is 4pm on release day. The build is green, the sprint board is clear, and someone asks the only question that matters: are we shipping this one, yes or no? Release testing is how a team answers that without guessing.

Release testing is the gate between a finished build and production. It runs against one specific candidate build, checks it against the specification rather than hunting for fresh coding defects, and ends in a decision that a named person owns.

Release testing verifies that a particular release of a software product meets its specified requirements and is ready to ship. It runs against a single candidate build, and it produces a go or no-go decision rather than a list of suggestions.

It is a form of system testing, with two distinctions that matter in practice:

• Independence: It is run by a team that was not involved in building the system, so the people deciding whether it ships are not the people who wrote the code.
• Intent: System testing by the development team hunts for defects. Release testing asks a narrower question: does this build meet its requirements well enough to go live?

In practice the suite combines regression testing, acceptance testing, and targeted performance and security checks. It is mostly black box: tests come from the specification, not from reading the implementation.

The obvious answer is that shipping a broken release is expensive. The more current answer is that code is now being produced faster than it is being validated.

In the 2025 DORA report, which drew on survey responses from nearly 5,000 technology professionals, 90% of respondents report using AI at work. The same research found that AI adoption continues to have a negative relationship with software delivery stability, even while it improves throughput. Google's DORA researchers describe AI as an amplifier: it magnifies whatever your pipeline already is.

That is the case for the gate, not against it. When more code arrives per sprint and stability is the metric under pressure, the checkpoint that decides what reaches users carries more weight, not less.

A release test is what turns that pressure into a decision. It confirms the build does what the specification says, exercises it against the operating systems and browsers your users actually run, and gives someone the evidence to say no when the evidence says no.

Release testing is easier to reason about once you know which stage a build is in. Each stage answers a different question, and the audience widens as confidence grows.

The distinction that trips teams up is alpha versus beta. Alpha is about an unfinished product and an internal audience. Beta is about a finished product and an unpredictable audience. Release testing comes after both, and it is deliberately narrow: by the RC stage, discovering a new feature request is not the point.

A release candidate is a build that is functionally complete, feature-frozen, and treated as shippable unless something blocking turns up. It is the specific artifact release testing runs against.

Two rules make the RC concept work, and both are commonly broken:

• The freeze is real: After a build becomes an RC, only fixes for blocking defects go in. A new feature slipped in after the freeze creates an untested build wearing a tested build's version number.
• The binary is promoted, not rebuilt: The artifact that passed is the artifact that ships. Rebuilding from source at deploy time means you shipped something release testing never saw.

Exit criteria are what turn an RC from an opinion into a decision. Write them before testing starts, because criteria invented at 4pm on release day are written by whoever most wants to ship. A workable set looks like this:

• Zero open blocking or critical defects against the RC build.
• The smoke testing suite passes on the RC, not on an earlier build.
• The regression suite passes on every browser and OS combination in your supported matrix.
• Performance and security checks sit inside agreed thresholds, with the thresholds recorded.
• A named owner signs off, and a rollback plan exists before anyone deploys.

Note what the list does not say: no cutoff expressed as a percentage of tests passing. A percentage target invites arguing about which failures counted while the clock runs. Blocking defects are binary on purpose.

A release suite is a bundle of checks, each answering a different failure question about the candidate.

• Functional testing: Does each feature do what the specification says, on the RC build specifically?
• Performance testing: Does response time and resource use hold under expected and peak load, measured against a recorded threshold rather than a feeling?
• Security testing: Do authentication, authorization, and encryption behave correctly, and does the release introduce a new exposure?
• Compatibility testing: Does it work across the browsers, operating systems, and devices your users actually run? This is the check most often skipped and most often regretted.
• Usability testing: Can a real user complete the primary journey without help?
• Regression testing: Did this release break something that used to work? Usually the largest part of the suite by test count.
• End-to-end testing: Does the whole system work together, in an environment that resembles production?

Unit tests belong earlier. By the time a build is an RC, unit tests have already run many times, and a unit failure at this stage means the pipeline let a broken commit through rather than that the release needs assessing.

Note: Compatibility is the release check teams cut first when the clock runs down, because a private device lab cannot cover the matrix. TestMu AI runs your release suite across 3,000+ browser and OS combinations and 10,000+ real devices, so the compatibility gate stays in the suite instead of becoming a post-release incident. Try it free

The sequence below assumes the RC exists and the freeze is in place.

• Write the test plan and exit criteria: Establish a test plan that states scope, timeline, and the explicit pass and fail conditions. Do this before the RC is cut.
• Stand up the environment: Build a testing environment that mirrors production closely enough that a pass means something.
• Prepare test data: Use test data shaped like production data, including the awkward records that break assumptions.
• Smoke first: Run the acceptance suite against the RC. If it fails, reject the candidate now rather than spending a day on deeper testing of a dead build.
• Execute the suite: Run test execution across the supported matrix and record results against the RC version, not against "latest".
• Triage against exit criteria: Use a defect management process to sort findings into blocking and non-blocking. Only the blocking list can stop the release.
• Make the call: Ship, or reject the candidate back to development. A rejected RC is release testing working, not release testing failing.

Step 4 is the one worth automating hardest, because it is the cheapest place to reject a bad candidate. A smoke check of that shape looks like the following, targeting a login journey on the TestMu AI Ecommerce Playground test site.

const session = await client.sessions.create({
  adapter: 'playwright',
  lambdatestOptions: {
    browserName: 'Chrome',
    browserVersion: 'latest',
    'LT:Options': {
      platform: 'Windows 11',
      build: 'Release Candidate Smoke Gate',
      name: 'RC smoke: login form reachable',
    },
  },
});

const { browser, page } = await client.playwright.connect(session);
await page.goto('https://ecommerce-playground.lambdatest.io/index.php?route=account/login');

// The gate: if the login form is not reachable, the candidate is dead on arrival.
const emailVisible = await page.locator('#input-email').isVisible();
const pwVisible = await page.locator('#input-password').isVisible();
console.log('SMOKE GATE :', emailVisible && pwVisible ? 'PASS' : 'FAIL');

await browser.close();
await client.sessions.release(session.id);

Running that against a live cloud session on 17 July 2026 produced this output:

SESSION ID: session_1784272091181_z46xsy
PAGE TITLE : Account Login
input-email visible    : true
input-password visible : true
SMOKE GATE : PASS
DURATION   : 20.1s

Twenty seconds is the useful part. A gate that answers in under a minute gets run on every candidate; a gate that takes an hour gets skipped on the candidate that most needed it.

A release test environment is a dedicated environment for validating a candidate, built to resemble production closely enough that results transfer. It needs:

• Infrastructure matching production: servers, network configuration, and databases of the same shape.
• Test data representative of production, including edge-case records.
• The supported browser, OS, and device matrix, which is where private labs run out of room.
• Isolation from development and production, so a test run cannot touch live data and a deploy cannot move the ground mid-test.

The failure mode is drift. An environment that quietly diverges from production turns a green release test into a false negative, and false negatives are worse than no gate because they carry authority.

This is where a cloud test infrastructure earns its place. TestMu AI orchestrates release suites through HyperExecute, and two of its controls exist for exactly this gate: fail-fast aborts the job after a set number of consecutive failures, so a uniformly broken candidate does not burn the whole suite's compute, and automatic retries absorb transient infrastructure failures so a flake does not read as a blocking defect. Both are YAML toggles:

# hyperexecute.yaml - gating controls for a release candidate
retryOnFailure: true
maxRetries: 2          # 1 to 5

failFast:
  maxNumberOfTests: 2  # abort after this many consecutive failures
  level: scenario

One caveat worth knowing before you rely on retries: they only fire when the test runner command itself exits non-zero. If your runner is configured to swallow failures and exit 0, HyperExecute sees a pass and no retry happens. The HyperExecute documentation covers the full YAML surface.

These are two phases with different questions, and conflating them is why some teams believe monitoring replaces testing.

Post-release testing does not make the gate optional. It catches the class of problem a test environment structurally cannot: real traffic patterns, real data volume, real third-party behavior on a Tuesday afternoon.

Some risk only exists in production. Two techniques let a team meet it deliberately instead of by accident.

Feature flags separate deploying code from releasing behavior. The new path ships to production turned off, gets enabled for a small group, and gets switched off the moment metrics degrade. The rollback is a config change rather than a redeploy, which is why it can happen in seconds.

Canary releases route a small slice of live traffic to the new version while everyone else stays on the old one, then compare error rates between the two. If the canary degrades, traffic shifts back before most users see anything. Our guide to canary testing covers the rollout mechanics in detail, and testing in production covers the wider practice.

Both depend on something teams forget to build first: the ability to detect degradation quickly and to reverse without a deploy. A feature flag with no monitoring behind it is not a safety mechanism, it is a hidden branch in production.

A gate with no owner is not a gate. In Agile and DevOps teams the traditional release manager has largely given way to a QA-led model, where the person closest to the evidence makes the call.

A QA release manager typically owns:

• Exit criteria: Defining them before the RC is cut, and refusing to renegotiate them under deadline pressure.
• Candidate promotion: Deciding when a build earns RC status and when the freeze starts.
• The quality gates in the pipeline: Which automated checks block a merge, and which only warn.
• Sign-off coordination: Getting QA, development, and operations to agree on go or no-go, with the disagreement recorded.
• The rollback call: Deciding when a live release is degrading enough to reverse, and owning that decision.

The tension in the role is structural: continuous delivery pushes for speed, and the gate exists to slow things down. That is not a flaw in the process, it is the job. The useful version of the role is not the person who says no, it is the person who made the criteria explicit early enough that nobody has to argue at 4pm.

Making that call needs evidence in one place: which tests ran against which candidate, what failed, and whether the failure is new. That reporting layer is what continuous testing feeds, and it is why release decisions get easier as pipeline visibility improves.

If you searched for release testing and landed on laboratory pages, this is why: the term means something specific and unrelated in regulated manufacturing, and both meanings share the SERP.

In pharmaceuticals, QC release testing is the regulated laboratory testing of a physical batch before it can be distributed. It is not a best practice, it is law. The US regulation at 21 CFR 211.165, titled "Testing and release for distribution", requires that for each batch of drug product "there shall be appropriate laboratory determination of satisfactory conformance to final specifications for the drug product, including the identity and strength of each active ingredient, prior to release". The same section requires that acceptance criteria be adequate to assure batches meet every appropriate specification as a condition of release.

The concepts rhyme, which is why the word is shared:

• Both gate distribution: Nothing reaches the customer until a defined check passes.
• Both need pre-agreed criteria: Software has exit criteria; a batch has final specifications set under Good Manufacturing Practice.
• Both reject rather than negotiate: A batch that fails its specification is rejected, in the same way a candidate with an open blocking defect does not ship.

Where they diverge is reversibility, and it is worth understanding if you build software for regulated industries. A software release can be rolled back in minutes with a feature flag. A distributed batch cannot be un-shipped, which is why the regulation puts the entire burden on testing before release and why the sampling plan itself has to be written down and followed.

Start with the exit criteria, not the tests. Write down what would make you refuse to ship, agree it before the candidate is cut, and the rest of release testing becomes a matter of gathering evidence against a list rather than arguing on release day.

From there the practical sequence is short: freeze the build, smoke it, run the regression suite across your real supported matrix, and let a named owner make the call with a rollback plan ready. Teams shipping on CI/CD run that same gate on every candidate, which is only sustainable when the suite is fast.

To put the gate into your own pipeline, run your release suite on TestMu AI's real device cloud so the compatibility check covers what your users actually run, and use the fail-fast and retry controls above to keep a broken candidate from burning an afternoon. The test strategy guide is a good next step for deciding which checks belong at the gate and which belong earlier.