Executable Specifications in Software Development and Testing

What Are Executable Specifications?

Executable specifications are requirements written so a machine can run them as tests. The requirement text names the test, worked examples supply the data, and a passing run proves the system still matches it.

You will also see them called executable requirements, a name that dominates in regulated and embedded work where each scenario has to be traced to a numbered requirement. Whichever label a team uses, three properties have to hold at once, and dropping any one of them produces something else:

• Readable by the requester - a product owner or domain expert can read the file and say whether it describes what they asked for, without a developer translating it.
• Runnable by a machine - a framework executes it and returns a pass or fail. A document that only a human can check is a requirement, not a specification in this sense.
• Versioned with the code - the file sits in the same repository and moves through the same pull request, so the specification and the implementation change together or the build stops.

The same requirement looks very different in each form. Here is one discount rule written first as prose and then as a runnable specification:

Executable specifications in software development grew out of two named practices. Dan North's behavior driven development supplied the Given, When, Then grammar, and Gojko Adzic's specification by example supplied the discipline of deriving every scenario from a concrete, agreed example rather than an abstract rule. The term living documentation describes the by-product: a suite that documents current behaviour because it is executed rather than reviewed.

If your team still writes requirements as prose first, the practical starting point is a well-formed software requirement specification and a habit of turning each numbered criterion into one worked example.

How Do Executable Specifications Work?

Executable specifications work through four separated layers: the specification in business language, a domain language that names the actions, a driver that talks to the running system, and the system itself under test.

Dave Farley popularised this four-layer split in continuous delivery work, and it is the single design decision that determines whether a suite survives two years of UI churn. Each layer knows only about the layer directly below it.

Teams that skip layers two and three write scenarios full of clicks and field names. When the login page is redesigned, every one of those scenarios fails at once and the specification has to be rewritten even though the business rule never changed. That is the failure mode behind most abandoned suites, and it is covered in more depth in these BDD pitfalls.

How Do Executable Specs Differ From Test Cases and Acceptance Criteria?

Acceptance criteria state the pass bar in prose, a test case lists steps for a person or script to follow, and an executable spec is the criterion itself written in a runnable form. The spec removes that hand-off.

These four artifacts get used interchangeably in most teams, which is why the same behaviour ends up documented in four places that disagree. They are distinguishable on four axes:

The practical consequence is that a specification suite collapses three of these into one artifact. Instead of writing a criterion, then a test case, then an automation script, the team writes the criterion once as a concrete example and binds it to code. Worked examples of that first conversion step are collected in these acceptance criteria examples.

Is an Executable Specification the Same as an Automated Test?

No. Every executable specification is an automated test, but most automated tests are not specifications. A test earns the name only if the person who asked for the behaviour can read it and confirm that it is correct.

Apply three checks to any test in your suite. It qualifies as a specification when all three pass:

• The read-aloud check - read the test name and its steps to a product manager. If they can tell you whether it is right, it is a specification. If they ask what a status code is, it is a test.
• The vocabulary check - the words in the test come from the business domain, not the technology. Cart, invoice, and refund qualify. Modal, endpoint, and payload do not.
• The traceability check - the test points back to a requirement someone actually asked for. A test with no owning requirement proves the code does what the code does.

Most suites contain both kinds and should. A unit test asserting that a date parser rejects the 30th of February is valuable and will never be a specification, because no stakeholder asked for it in those words. The mistake is expecting a folder of assertion-level tests to serve as documentation, and then wondering why nobody outside engineering ever reads them.

How Do BDD and Gherkin Turn Requirements Into Runnable Specs?

Behavior driven development runs a conversation that produces concrete examples, and Gherkin writes them in the Given When Then pattern. A step definition file then binds each line to code that drives the application.

Gherkin is a small, deliberately limited grammar. The Gherkin reference defines the full keyword set, but four of them carry almost every scenario you will write:

• Given - the state of the world before the behaviour happens. Put setup here, never actions the user performs.
• When - the single action being specified. One When per scenario keeps the failure message unambiguous.
• Then - the observable outcome. Assert on something the user could see, not on a database row or an internal flag.
• Scenario Outline - the same scenario run against a table of Examples, which is how one specification covers a dozen boundary values without a dozen files.

A Scenario Outline is the highest-value construct in the language and the most commonly skipped. This one specification replaces four near-identical scenarios and makes the boundary at 100 dollars visible at a glance:

Feature: Checkout discounts

  Scenario Outline: Free shipping applies above the threshold
    Given a shopper has a cart worth <cart> dollars
    When they proceed to checkout
    Then the shipping charge is <shipping> dollars

    Examples:
      | cart | shipping |
      |   99 |        5 |
      |  100 |        0 |
      |  101 |        0 |
      |    0 |        0 |

The last row is the one that earns the table. A cart worth nothing should probably not qualify for free shipping, and writing the example is what forces someone to decide. Teams new to the grammar usually get more value from reading real Gherkin test cases than from the keyword list, and the wider practice split is covered in TDD vs BDD.

How Do You Write an Executable Specification, Step by Step?

Start from one acceptance criterion, write a concrete example with real values, phrase each step as an observable outcome rather than a click, bind the steps to a driver, and run it in the pipeline that gates merges.

The five steps below take a single requirement all the way to a green run. The example is deliberately small so the mechanics stay visible.

• Pick one criterion - one rule, one scenario. A scenario covering three rules produces a failure message that tells you nothing about which rule broke.
• Replace every abstraction with a value - a valid message becomes the exact string you will type. This is the step that surfaces disagreement, and it costs minutes rather than a sprint.
• Write declaratively - the step says what happened, not which control was used. Submits their message survives a redesign; clicks the button with id showInput does not.
• Bind steps to a driver - the step definition is the only file that names selectors, and it should read as a thin translation rather than as test logic.
• Run it where merges are gated - a specification that runs only on a laptop documents one developer's machine, not the product.

Here is the specification for a single-input form on the TestMu AI Selenium Playground. Note that no step mentions an element:

Feature: Simple form submission

  Scenario: A single input field echoes the submitted message
    Given the user is on the Simple Form Demo page
    When the user submits the message "Executable specs are runnable requirements"
    Then the page displays that exact message

The step definitions carry every selector, and the cloud capabilities live beside them so the same specification can run locally or on a grid without a single edit to the feature file:

const { Given, When, Then } = require('@cucumber/cucumber');
const { chromium, expect } = require('@playwright/test');

const capabilities = {
  browserName: 'Chrome',
  browserVersion: 'latest',
  'LT:Options': {
    platform: 'Windows 11',
    build: 'Executable Specifications Blog',
    name: 'Scenario: A single input field echoes the submitted message',
    user: process.env.LT_USERNAME,
    accessKey: process.env.LT_ACCESS_KEY,
  },
};

Given('the user is on the Simple Form Demo page', async function () {
  this.browser = await chromium.connect(
    'wss://cdp.lambdatest.com/playwright?capabilities=' +
      encodeURIComponent(JSON.stringify(capabilities))
  );
  this.page = await this.browser.newPage();
  await this.page.goto('https://www.testmuai.com/selenium-playground/simple-form-demo/');
});

When('the user submits the message {string}', async function (message) {
  this.message = message;
  await this.page.fill('#user-message', message);
  await this.page.click('#showInput');
});

Then('the page displays that exact message', async function () {
  await expect(this.page.locator('#message')).toHaveText(this.message);
  await this.browser.close();
});

Those three steps were executed against the Simple Form Demo page on the TestMu AI cloud grid on August 26, 2026. The run log below is the real output, and the screenshot is the page state at the moment the Then step asserted:

[Given] Simple Form Demo page loaded -> Selenium Grid Online | Run Selenium Test On Cloud
[When]  message entered and submitted
[Then]  displayed message = "Executable specs are runnable requirements"
RESULT: 1 scenario (1 passed), 3 steps (3 passed)

Notice what the feature file does not contain: no selector, no URL, no wait. Redesign that form and only the step definition changes. The same separation applies when the driver is Playwright rather than Selenium, which is walked through in this guide to Playwright with Cucumber.

Where Do Executable Specs Belong in the Test Pyramid?

They usually live at the acceptance layer, but the same spec can be driven at unit, integration, and acceptance level by swapping the protocol driver. Running it low gives speed, and running it high gives confidence.

This is the part almost every guide skips, and it follows directly from the four-layer split. Because layer three is the only layer that knows how to reach the system, one feature file can be pointed at three different entry points.

Reusing one specification across two layers costs a second driver and buys much earlier feedback on the same business rule. Reusing it across all three is rarely worth the maintenance, so most teams pick the unit layer for rule-heavy domains and the acceptance layer for journey-heavy products. Where those journeys cross acceptance testing boundaries, keep the specification identical and let only the driver change.

How Do You Run Executable Specs in a CI/CD Pipeline?

Run them as a required check on every pull request. Tag the fast specs to run on commit and the full suite on merge, publish the report as a build artifact, and fail the job on any undefined or pending step.

Executable specifications testing only pays for itself when a failing scenario can block a merge. A suite that runs nightly and gets ignored by 10am is documentation with a cron job attached. Four settings do most of the work:

• Strict mode - fail the run when a step has no definition or is marked pending. Without it, a half-written scenario reports green and the requirement it describes is silently unverified.
• Tag-based selection - tag scenarios by speed and risk, then run the smoke tag on every push and the full set on merge to the default branch.
• Parallel execution - split by scenario rather than by file, because one long feature file otherwise sets the floor for the whole job.
• Published reports - upload the HTML or JSON report as an artifact so a product owner can read the failing scenario without asking an engineer to interpret a log.

name: executable-specs
on: [pull_request]

jobs:
  specs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - name: Run the specification suite
        env:
          LT_USERNAME: ${{ secrets.LT_USERNAME }}
          LT_ACCESS_KEY: ${{ secrets.LT_ACCESS_KEY }}
        run: npx cucumber-js --strict --tags "@smoke" --parallel 4 --format html:reports/specs.html
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: specification-report
          path: reports/specs.html

The --strict flag is the one teams most often leave off, and it is the difference between a suite that reports coverage and a suite that has it. Scaling the same idea further is a matter of infrastructure rather than authoring, which is where a cloud grid replaces a queue of local runs.

How Do Executable Specs Work With AI Coding Agents?

An AI coding agent reads and writes source code, so it cannot see whether the rendered page actually works. An executable spec gives the agent a machine-readable pass or fail that it can check its own output against.

The volume problem is now measurable. GitHub's Octoverse 2025 report counts 43.2 million pull requests merged each month, up 23% year over year, alongside nearly a billion commits in 2025. Review capacity did not grow 23%.

An agent's verification primitives are unit tests, type checkers, linters, and compilers, and every one of them operates on the same closed surface: text. None of them render the application, click a button, or confirm that a redirect lands on the right URL. That produces a specific failure class where the code is correct and the user-facing result is broken.

A specification written before the agent starts changes the loop in three ways:

• The prompt gets a pass bar - the agent is told what done means in observable terms, instead of inferring it from a ticket title.
• The verdict stops being self-reported - the agent runs the specification and reports an external result, so passed means a scenario went green rather than the model believing it did.
• Regression is checked automatically - an agent that changes 200 lines in one prompt is checked against every rule the team has ever written down, not just the one it was asked about.

Kane CLI is built around exactly this loop. It ingests a source of intent such as a PRD or a ticket, derives business use cases, scenarios, and acceptance criteria, and writes each one to a committable plain-English test.md file whose every step is wired to the criterion it proves. Running it drives a real Chrome browser and seals an evidence pack containing per-step screenshots, a HAR network log, and console output.

The coverage command is the part that matters for specifications. It reports which acceptance criteria were actually proved by the evidence rather than how many steps ran, and its strict flag demotes a green that was earned against a specification that has since changed. A number that is allowed to go down is one you can trust when it goes up. The Kane CLI getting started guide covers installation and the first run.

# derive use cases, scenarios and acceptance criteria from a requirement
kane-cli context ingest ./docs/checkout-prd.md
kane-cli design

# run the generated plain-English specification against a real browser
kane-cli testmd run ./tests/checkout_test.md

# report what the evidence actually proved, per acceptance criterion
kane-cli cover --strict

The broader shift toward agents owning parts of the quality loop is covered in this piece on agentic QA.

What Are the Benefits of Executable Specs, and How Do You Measure Them?

The measurable benefits are fewer requirement defects found late, a regression suite that costs nothing extra to build, and documentation that cannot go stale because a wrong sentence turns the build red.

Every benefit below has a mechanism and a metric attached, because a benefit you cannot measure is a preference. Baseline each one for a quarter before adopting the practice so the comparison means something:

The last row is the one to lead with when asking for time to adopt this. Line coverage rises while risk stays flat, because executing a line is not the same as proving a requirement. Coverage measured per acceptance criterion is the only number that answers which rules are unverified, and it is the number a release conversation actually needs.

When Are Executable Specs the Wrong Choice?

Skip them when no non-programmer will ever read the specification, when the behaviour is easier to state in code than in prose, or when nobody outside engineering will join the conversation that produces the examples.

The practice has a real cost, and every competing guide on this topic leaves it out. Four situations where the cost exceeds the return:

• No business reader - on an internal library or a compiler, the stakeholder is a developer. A well-named test in the host language communicates faster than a Gherkin translation layer nobody asked for.
• Algorithmic behaviour - matrix operations, encoding rules, and numerical tolerances are more precise as code. Forcing them into Given, When, Then loses precision and gains nothing.
• Solo or engineer-only teams - the value comes from the conversation between roles. Without a product or domain voice in the room, you have added a parsing layer to your test suite and no shared understanding.
• Exploratory and non-functional work - usability judgement, exploratory sessions, and load-profile analysis produce findings, not pass and fail verdicts. Keep those outside the specification suite.

One further warning applies even to teams that should adopt the practice: step definition sprawl. Ten near-identical phrasings of the same action produce ten bindings that drift apart, and the suite becomes harder to change than the code it guards. Enforce a shared step vocabulary in review from the first scenario, and treat a new step phrasing as a design decision rather than a convenience. The formal variant of this discipline is covered under acceptance test driven development.

Key Takeaways

Conclusion

Pick the single acceptance criterion that caused the most rework in your last release and rewrite it as one scenario with real values in it. Bind it to a driver, add it to the pull request check, and see whether the team agrees on what the numbers should be. That one file will tell you more about your requirements process than a month of refinement meetings.

If the binding layer is the blocker, TestMu AI's KaneAI authors and maintains the executable half from natural language while the specification stays yours, and self-healing re-anchors steps when the interface changes instead of failing on a stale selector. The KaneAI documentation walks through authoring a first test from a requirement, and the wider practice of moving verification earlier is covered in shift left testing.