Introducing Test.md: Kane CLI's Framework for Replayable Tests

Test.md Is a Framework. The Syntax Is Markdown.

Test.md is Kane CLI's test framework. It defines how a test is structured, how steps connect, how variables flow, how files compose, and how replays execute. Everything a framework provides.

What it does not provide: selectors, locators, page objects, fixtures, assertions in code, runner configs, or a test grammar to memorize.

The syntax is markdown. The structure is heading and objective pairs. The runtime is Kane CLI.

This is the first test framework where the test file reads like documentation, because it is documentation.

Every Kane CLI Session Can Become a Test

You can author a _test.md by hand, or record one from a live session. Pass --name to any run:

kane-cli run "Search for noise-cancelling headphones on amazon.com" --name amazon-search

The browser executes. The session records. On exit, Kane CLI writes the session out to <cwd>/.testmuai/tests/amazon-search_test.md. Move it into your repo. Commit it. The file is now a regular test.

This is the path of least resistance from "I am debugging this issue right now" to "this issue is permanent regression coverage in CI." No second pass. No rewriting. The exploration is the test.

Step Headings. Plain English Objectives.

Test.md uses markdown's own structure. The convention is small and consistent. Tests live in files ending in _test.md.

---
mode: testing
max_steps: 35
---

# Amazon, add to cart

## Open Amazon
Open https://www.amazon.com.

## Search for headphones
Type "wireless headphones" into the search box and submit.
Verify at least one product result is visible.

## Open the first result
Click the first non-sponsored product card. Verify the product detail page loads.

## Add to cart
Click "Add to Cart". Verify the cart icon shows a count of 1 or higher.

Double hash marks a step. The objective beneath is plain English. One objective per heading.

That is the syntax. The whole syntax.

Headings double as documentation. Anyone reading the file knows what the test covers without running it. Anyone editing the file knows where to insert, remove, or reorder steps. Pull requests show step-level diffs in prose, not in selector strings or method chains.

Run the test with:

kane-cli testmd run amazon_test.md

Note: Read the Kane CLI documentation and start writing replayable tests today. Click here

Imports Compose Flows Without Copying Them

Tests get long. Login flows repeat. Setup steps repeat. Cleanup steps repeat.

Test.md handles repetition with a single directive.

## Sign in
@import ../helpers/login.md

## Open the new dashboard
Click Settings, click Beta features, assert the page contains 'Admin Console'.

## Tear down
@import ../helpers/sign-out.md

Helpers are markdown files that do not end in _test.md. They can only be reached through @import, never run directly. The imported file runs in place. Its steps execute as if they lived in the importing test. Variables defined upstream stay in scope.

Imports nest. A shared login helper can import a shared environment-setup helper. Compose flows the way you compose code, without duplicating objectives across files.

Path resolution is relative to the importing file, never to your shell. The same helper layout works on any teammate's machine and in any CI runner.

The practical result is a helpers/ directory of building blocks. Login. Logout. Seed data. Reset state. Every test pulls from it. Update a helper once and every test that imports it picks up the change on the next run.

One File. Every Environment.

A _test.md is more than its steps. It carries its own configuration. Front matter at the top of the file. Variables, runtime limits, browser routing, run mode.

---
mode: testing
max_steps: 50
timeout: 90
target: chrome
headless: false
variables:
  app_url: "https://staging.app.com"
  admin_email: "[email protected]"
  admin_password:
    value: "s3cret-pa55"
    secret: true
---

## Login
Go to {{app_url}}, fill the email with {{admin_email}}, ...

The keys you will reach for most:

• Variables. Anything in {{double_braces}} resolves at runtime. Mark a variable secret: true and the value is masked in logs, redacted in Result.md, and routed to the secrets store instead of being synced as a plain Test Manager variable. The same test runs against dev, staging, and prod. Swap variables, not test logic.
• Runtime limits. max_steps caps how many agent actions a single step can take. timeout is a hard kill timer in seconds. Set them globally. Override per step when one step legitimately needs more room.
• Browser routing. target picks the execution surface: chrome for managed local Chrome, cdp to reuse an external Chrome over cdp_endpoint, or ws for a TestMu AI or Playwright WebSocket at ws_endpoint. headless runs Chrome without a window. Local, cloud, tunnel, all from the same file.
• Run mode. mode: testing (default) lets the agent push through auth walls and error pages so negative-test assertions work. mode: action halts on those pages so a human can intervene.

Step-level overrides handle the long tail. Drop a fenced yaml block under any step heading to change max_steps, timeout, or variables for that step alone. The defaults at the top are defaults, not constraints.

Replay Is Automatic. So Is Re-Authoring.

A test file you cannot rerun is a script. Test.md is replayable on every run, and the replay logic is built into the runtime.

The first time you run a _test.md, every step is authored: the agent works out how to satisfy the objective and a recording is saved next to the file. On every run after that, Kane CLI replays the cached recording for each step. No agent. No LLM cost. Much faster.

A step replays if its objective and yaml block are unchanged since the recording. Edit a step and that step authors again. Edit step 3 of a 10-step test and steps 3 through 10 re-author, because step 4 expected to start where step 3 left off.

This is by design. The cache stays honest. You never replay against state that no longer exists.

Output lands in output-<stem>/ next to the test file:

amazon_test.md
output-amazon/
  Result.md
  .internal/                # cached recordings, do not edit
  playwright-python-code/   # if code export is enabled

Commit the whole output-<stem>/ directory. Tests now replay deterministically on any teammate's machine and on CI, no re-authoring required.

Result.md is the human-readable run report. Step-by-step status with timings, the shareable evidence link, the full trace. Drop it in any markdown viewer.

Built for CI From Day One

A CI-friendly invocation:

kane-cli testmd run ./tests/checkout_test.md \
  --agent \
  --headless \
  --on-lock-conflict wait \
  --retry

• --agent streams NDJSON to stdout. One JSON object per line. Parseable by any pipeline.
• --headless runs Chrome without a window.
• --on-lock-conflict wait blocks if a teammate is editing the same test, instead of failing the build.
• --retry recovers transient replay failures by re-authoring the failing step with a shrinking replay window.

Exit codes are standard. 0 passed. 1 non-optional step failed. 2 error before or during the run. 3 cancelled or timed out. Drop into any shell, any pipeline.

Native Playwright, Both Ways

Every successful run can produce runnable Playwright code in Python or JavaScript.

kane-cli testmd run amazon_test.md --code-export --code-language python

The generated project lands in output-<stem>/playwright-python-code/. Take it. Modify it. Run it under your existing Playwright setup. Or set code_export: true in front matter and have it generated on every run.

Then hand it back. Kane CLI now reads its own Playwright exports and runs them. Edit the exported test in your IDE, refactor, add assertions, and pass the file straight back to the runtime. Round-trip without rewriting either side.

You start in plain English. You can leave with native browser-automation code. And you can come back when you want the agent in the loop again. The framework is a two-way door.

Agents Finally Have a Test Format

AI coding agents have a verification problem. They write code. They can run it. They cannot keep the verification.

Test.md fixes that.

An agent builds a feature. The agent calls Kane CLI to verify the feature works in a real browser. Kane CLI writes a _test.md and saves the recordings. The agent commits both alongside the feature code.

The next agent run reads the _test.md files first. It knows what should still work, because the last agent left receipts. Regressions surface immediately. Features arrive with proof attached.

This is what agent-native actually means. Not "an agent can call the CLI." It means the agent and the human share a single, durable, version-controlled artifact that describes what the application is supposed to do, in language both can read and both can write.

Test.md is the framework agents have been missing. Markdown, because every agent reads markdown. Plain English objectives, because every agent generates English. File-based, because every agent commits files.

Note: Building with AI agents? Point your agent here: testmuai.com/kane-cli/agents.md

Humans Stop Choosing Between Exploring and Testing

Manual testing and automated testing have always been two different jobs. You explored to find issues. You wrote tests later, separately, often weeks later, often not at all.

Test.md collapses the gap.

You explore in Kane CLI with --name. The session records. On exit, the test exists in .testmuai/tests/. Move it into your repo and it is part of the suite.

The 2 AM debug session is regression coverage by morning. The customer bug repro is a permanent check before the ticket closes. The "let me just try one thing" turns into a _test.md you commit.

Your test suite grows the way your codebase grows. Linearly with the work you were going to do anyway. Not as a separate project that lives in someone else's quarterly OKRs.

Get Started

Update Kane CLI.

npm install -g @testmuai/kane-cli

Or

brew install LambdaTest/kane/kane-cli

Record a test from a session:

kane-cli run "Verify the checkout flow" --name checkout

Run it again, anywhere, anytime:

kane-cli testmd run checkout_test.md

Building with AI agents? Point your agent to testmuai.com/kane-cli/agents.md

Start for free at testmuai.com/kane-cli.