From PRD to Evidence: the Kane CLI 0.6 Release

The Shape of the Thing

Everything Kane CLI 0.6 knows lives in one append-only, content-addressed graph under .context/. It has six node types, and they form a chain.

source -> usecase -> ac -> scenario -> test
                                    -> gap

context ingest creates a source. context extract derives use-cases from it. design tests derives acceptance criteria, then scenarios, then tests, strictly 1:1 with scenarios. When the design needs something it cannot invent, it records a gap rather than guessing.

That last node type is the tell. Most test generators, asked to test a checkout flow without being given a URL or a card number, will cheerfully hallucinate both. Kane CLI writes down that it does not know.

The Loop, Run for Real

Here is an actual run against a billing spec: three plans, an upgrade flow with declined and expired card rules, a downgrade with a project-count guard, and an invoice list.

kane-cli context ingest specs/billing.md

created  billing  source sha256:1fa4af8e...  blob sha256:35c9715f...

kane-cli context extract --mode agent

extracting: billing @ sha256:1fa4af8e...
  - trace provenance...
  - search graph...        (x4, checking possible duplicates)
  - propose use-cases...
committed: 0 trusted, 4 derived (queued)

uc-review-subscription-plans-and-pricing
uc-upgrade-from-free-to-pro
uc-downgrade-from-pro-to-free
uc-view-past-invoices

kane-cli context list

trust . fresh . title . id

-- source . 1 --
  -        fresh  billing                                  billing

-- usecase . 4 --
  derived  fresh  Downgrade from Pro to Free               uc-downgrade-from-pro-to-free
  derived  fresh  Review subscription plans and pricing    uc-review-subscription-plans-and-pricing
  derived  fresh  Upgrade from Free to Pro                 uc-upgrade-from-free-to-pro
  derived  fresh  View past invoices                       uc-view-past-invoices

4. Design

kane-cli design tests --use-case uc-upgrade-from-free-to-pro --mode agent --max 4

This is the heart of the release. One committed use-case goes in; acceptance criteria, scenarios, and tests come out. --max is a real ceiling: the agent runs an explicit trim-to-budget pass to hit it, and omitting it means no ceiling, at which point the agent estimates the size and asks you. Always set it.

The run produced 18 acceptance criteria, 4 scenarios, 4 tests, and 3 gaps for about 92 credits, in phases, with a receipt after each.

committed 1 AC (derived), approvals promote via chat
receipt: parity { live_scenarios: 4, live_tests: 0, missing: [ ... ] }
next: emit tests for the 4 scenario(s) listed, 1:1, strict.
...
committed 4 tests (derived)
receipt: parity { live_scenarios: 4, live_tests: 4, missing: [] }
next: every live scenario has a test.

It also validates its own drafts and retries. Midway through, the first proposal failed validation and the agent refined it rather than committing something malformed.

  - propose design -> validation failed -> refining the draft -> propose design
  - propose design... -> propose design

And it wrote four runnable files.

wrote: .testmuai/tests/t-upgrade-from-free-to-pro-succeeds-and-updates-the-account_test.md
       .testmuai/tests/t-current-plan-card-is-gated-with-badge-and-no-upgrade-cta_test.md
       .testmuai/tests/t-declined-card-keeps-checkout-open-and-does-not-upgrade-the_test.md
       .testmuai/tests/t-expired-card-is-rejected-in-the-browser-before-any-charge_test.md

Note: Point Kane CLI at what your product must do and let TestMu AI design the tests that prove it. Start for free

---
assurance:
  id: t-declined-card-keeps-checkout-open-and-does-not-upgrade-the
  base: sha256:e890c5d2...
---
# Declined card keeps checkout open and does not upgrade the account

> Prove that a declined card response keeps the checkout modal open, shows the
> inline decline message, leaves the account on Free, and does not trigger an
> automatic retry.

## Step 4
Before confirming payment, clear DevTools Network so only requests caused by this
confirmation attempt are captured.

## Step 7
In DevTools Network, store the number of payment or charge confirmation requests
triggered by that single Confirm action as decline_charge_attempt_count and verify
that only one such request was sent.

## Step 9, assert @verifies ac-a-declined-card-keeps-the-checkout-modal-open, ...
Confirm absolute check: inline decline error text is "Card declined, try another
card" (equals), the stated promise: A declined card shows the inline error
message "Card declined, try another card".

Coverage vs the Feeling of Coverage

Design finished and then said this, four times.

warning: t1: verifies 9 ACs but machine-asserts only 1
         (check.verified_against=sha256:c8492d34...), the rest ride prose only
warning: 28 verifies edges target not-yet-approved items, committed anyway

The test tags nine acceptance criteria and mechanically proves exactly one. The other eight ride on prose steps that an agent interprets at runtime. That is not a bug, it is often the honest shape of a browser test. But most tools would have let you believe you had nine criteria covered.

This is the difference between coverage and the feeling of coverage, and 0.6 refuses to let you confuse the two. Because each test records the criteria it covers, results report per criterion rather than per test: a test satisfying three of five tagged ACs shows exactly that, instead of one green check mark that means nothing in particular.

Gaps: the Design Tells You What It Lacks

kane-cli context list --type gap

-- gap . 3 --
  derived  fresh  Need browser start URL      gap-need-browser-start-url
  derived  fresh  Need Free-user login        gap-need-free-user-login
  derived  fresh  Need payment fixtures       gap-need-payment-fixtures

Those three gaps correspond exactly to the twelve {{variables}} left unbound across the four tests: pricing_url, free_user_email, declined_card_number, and so on. The design refused to invent a URL or a card number, recorded what it needed as first-class nodes, and handed you a setup list. Bind them with --variables-file and the suite is live.

Every Node Remembers Its Own Argument

Two commands, no model calls, no cost.

kane-cli context explain t-declined-card-keeps-checkout-open-and-does-not-upgrade-the

#6  minted     test sha256:5d563760...
#6  judgement  why: [value_score] Kept at score 9 because declined-payment handling
               is a high-risk failure path with potential billing and state
               integrity impact.
#6  name       named t-declined-card-keeps-checkout-open-and-does-not-upgrade-the

kane-cli design explain t-declined-card-keeps-checkout-open-and-does-not-upgrade-the

technique: negative for payment failure behavior; state_flow for preserving the
           in-flow modal state after failure; idempotency profile applied to the
           no-auto-retry promise. No covering_array applied because the outcome is
           driven by a single declined-payment partition; risk-judged interaction
           strength would be 3 for money/auth if multiple independent inputs
           required combination coverage. Budget fit: kept within ceiling 4.
combinatorial: strength 2
check: absolute equals inline decline error text is "Card declined, try another card"
automates: sc-declined-card-leaves-checkout-open-without-plan-change
verifies: ac-a-declined-card-keeps-the-checkout-modal-open  (+8 more)

This is not the model re-rationalising after the fact. These are the judgements recorded at design time, replayed from the chain. Ask why does this test exist, why is it shaped this way, and what does it actually prove six months from now, and you get the same answer you would have gotten on day one.

The Chain Is the Truth

.context/ is an append-only commit chain. History is never rewritten.

• context revert <seq> inverts a record's effects with a compensation record. The original stays.
• context retire <source_id> writes a tombstone. Reversible via revert.
• context fsck verifies the full chain and checks the derived projection for drift.

ok: 6 record(s) verified, derived projection in parity

context rebuild wipes .context/derived and regenerates it from the verified chain. That is the mental model worth internalising: .context/derived is a disposable cache; the commit chain is the source of truth. Never hand-edit under .context/.

Guards That Cost Nothing

Two of our favourite behaviours in this release are refusals, and both are free, they exit before a single credit burns. Re-extract an unchanged source:

{"type":"corpus","sources":[],"skipped":[{"source_id":"billing","reason":"cached"}]}
extract: nothing to do, every ingested source is already extracted

Redesign a use-case that already has a live design:

design: 'uc-upgrade-from-free-to-pro' is already designed @ v1, current;
        use --force to redesign
{"type":"gate_refused","verb":"design","exit_code":2}

That gate is not in your way. It is protecting ninety credits of work from a stray arrow-up.

Staleness Follows Lineage

Append a ## Refunds section to the spec and re-ingest.

kane-cli context ingest specs/billing.md    # versioned
kane-cli context list --stale

-- usecase . 4 --
  derived  stale  Downgrade from Pro to Free
  derived  stale  Review subscription plans and pricing
  derived  stale  Upgrade from Free to Pro
  derived  stale  View past invoices

All four went stale, including invoices and downgrade, which the refunds edit never touched. Staleness follows lineage from the source head, not the diff. This is deliberate and conservative: the graph will not quietly assure you that an untouched branch is still valid when the document beneath it moved. The practical consequence is that source granularity is a design decision. One giant PRD means every edit dirties everything; scoped sources keep the blast radius small.

Run It, and Seal It

Designed tests run through the same surfaces you already know: testmd run for one, testrun run for many as a single sealed execution, and every run seals exactly one evidence pack, a .evidence zip in .testmuai/evidence/.

<execution_id>.evidence
|-- run.yaml                     # manifest: title, status, timings, totals
|-- failure.yaml                 # run-level failure rollup
|-- tests/<test-id>/
    |-- test.md                  # the definition
    |-- result.yaml              # verdict + steps[]
    |-- logs/                    # run log, actions NDJSON, console NDJSON, network HAR
    |-- steps/<n>-<step-id>/     # screenshot.png, annotated.png, step.json
    |-- auteur/execution.json    # full execution trajectory
    |-- v16-trajectory/          # per-run planning summaries

The pack is the only place run artifacts live; the legacy runs/<n>/ directory is gone. kane-cli evidence serve <pack> opens a local-only viewer (nothing uploads; your browser reads the pack off your machine), evidence validate gates integrity in CI with exit codes (0 valid, 1 invalid, 2 not found), and evidence merge --run-id <id> combines packs into one.

So the chain closes: a line in a PRD becomes an acceptance criterion, which becomes a scenario, which becomes exactly one test, which produces a screenshot, a network log, and a verdict, and every hop between them is recorded and replayable. Every run also lands in TestMu AI Test Manager, so the evidence outlives the terminal it ran in.

When the Product Changes

kane-cli maintain reconcile     # re-ingest + re-extract a changed source, triage item by item
kane-cli maintain evolve <ref>  # re-design a node's parent use-case, report the pair diff
kane-cli maintain learn         # what the review loop has learned about your style

reconcile is the loop closing on itself: the spec moved, the graph knows which nodes went stale, and you walk the changeset one item at a time instead of regenerating a suite from scratch and diffing 400 files by hand. learn is the quiet one: the review loop distils your verdicts into style notes, so the design agent argues less with you over time.

Note: Want the full command reference for context, design, and maintain? Read the Kane CLI documentation. View the docs

What Is Still Rough

A launch post that only lists wins is a brochure. Three honest edges from the run above.

• Designed tests are not immediately batch-runnable. Straight out of design, testrun run ... --dry-run reports missing_meta and valid: false for every member, with has_meta: false and synced: false. Each test needs one testmd run to register before testrun will batch it. Use --dry-run; it is free and instant, and it will tell you this in under a second.
• The context extract --apply flag is not implemented yet. --plan prints a proposal and commits nothing, but its partner refuses loudly today. So --plan is for reading, not for a plan-then-apply gate in CI. Build that gate on context review instead.
• The --help output understates context list --type. It documents source | usecase; the graph actually filters on all six labels, gap included. The help text is behind the implementation.

And one number worth stating plainly: designing one use-case cost about 100 credits end to end. Four would run about 400. This is a deliberate, deep, self-validating pass, not a template expansion, and it is priced like one. Scope your --max, and design the use-cases that carry risk first.

Get It

npm install -g @testmuai/kane-cli@latest      # npm
brew install lambdatest/tap/kane-cli          # Homebrew
curl -fsSL https://raw.githubusercontent.com/LambdaTest/kane-cli/main/install.sh | bash

Upgrading from 0.6.0? Do it now. 0.6.0 shipped without a component that context, design, and maintain require, so those commands failed immediately after install; 0.6.1 bundles it correctly on every platform and stops a missing binary from aborting the install itself.

0.6.2 then landed the correctness pass this post is written against: OAuth tokens now exchange for long-lived credentials automatically so sessions no longer break mid-run, condition blocks evaluate logical and boolean expressions, SET variables resolve from the store instruction at runtime instead of being pre-seeded, empty expected values no longer fail non-contains assertions, and visual checkpoint conditions inherit from the right base.

The Point

The old promise was describe a test and we will run it. The new one is harder and more useful: describe your product, and we will tell you what proves it, and what does not. A test that tags nine criteria and checks one now says so out loud. A design that needs a card number says so instead of inventing one. A judgement made in July is still legible in December, because the graph kept the receipt.

Point Kane CLI at a spec you actually care about. Kane CLI 0.6.2 is available now.