Run End-to-End Tests from Your Terminal with Kane CLI
Author and Run Tests Right from the Terminal in Natural Language, with an Evidence-Backed Pass or Fail from a Real Browser.
npm install -g @testmuai/kane-cli
- TestMu AI (Formerly LambdaTest)
- /
- Blog
- /
- Spec-Driven Development Tools Compared: Spec Kit vs OpenSpec vs Kiro vs BMAD
Pick a spec-driven development tool from its README alone and the leading options sound alike: each turns a request into requirements, a design or plan, and tasks that a coding agent builds from. The differences show up later, in the unit of work, the requirement format your reviewers read, the coding agents it runs in, and what it checks after the code is written.
This comparison reads each tool's own repository and documentation, as of November 2026, and shows the files each one writes. TestMu AI builds testing tools rather than spec tools, so none of its products is ranked here; the testing section covers where a test step fits after any of the four. Readers new to the method can start with what spec-driven development is.
Overview
Choose GitHub Spec Kit to plan new features inside the coding agent you already use, or OpenSpec to plan small, reviewable changes to code that already exists. Choose Kiro if your team will work inside Amazon's IDE, CLI or web app, and the BMAD Method when a product needs a brief, a PRD and an architecture before stories are built.
What Are the Best Spec-Driven Development Tools?
- New features in your own agent: GitHub Spec Kit takes each feature through constitution, spec, plan, tasks, implementation and convergence, and its specify CLI sets up any of 41 named coding agents. Open source: yes, MIT.
- Changes to an existing codebase: OpenSpec puts each change in its own folder of proposal, design, tasks and spec deltas, then merges the deltas into the main specs when the change is archived. Open source: yes, MIT.
- One integrated tool: Kiro, from Amazon, writes requirements.md or bugfix.md, design.md and tasks.md for every spec in its IDE, CLI or web app, and its IDE can generate property-based tests. Open source: no, licensed as AWS Content.
- Product-scale planning: The BMAD Method's stable v6.12.4 installs five named agents, from Analyst to Developer, and sizes planning from a short SPEC.md to a brief, a PRD and an architecture. Open source: yes, MIT.
- Proof against the spec: TestMu AI's Kane CLI can ingest Markdown spec files from Spec Kit, OpenSpec, Kiro or BMAD, design one browser test per scenario tagged with the acceptance criteria it verifies, and report which criteria a run proved.
Which Spec-Driven Development Tool Should You Use?
Choose by the unit of work you plan in: Spec Kit plans per feature, OpenSpec per change, Kiro per spec inside its own tool, and BMAD per intent, from one SPEC.md up to a product with epics. The other rows follow from that choice, starting with the files your reviewers will read.
| Criterion | GitHub Spec Kit | OpenSpec | Kiro | BMAD Method (v6.12.4) |
|---|---|---|---|---|
| What it is | Open-source toolkit plus the specify CLI | Open-source spec framework plus the openspec CLI | Amazon's agentic IDE, CLI and web app with specs built in | Open-source agile method: named agents plus workflow skills |
| Unit of work | A numbered feature folder under specs/ | A change folder under openspec/changes/ | A spec folder under .kiro/specs/ | Sized to the work: one SPEC.md up to a PRD with epics |
| Main files | spec.md, plan.md, tasks.md, research.md, data-model.md, contracts/ | proposal.md, design.md, tasks.md, delta specs | requirements.md or bugfix.md, design.md, tasks.md | brief.md, prd.md, ARCHITECTURE-SPINE.md, SPEC.md, epics.md |
| Requirement style | Given/When/Then scenarios; FR-001 MUST lines; SC-001 outcomes | Requirements with SHALL or MUST; WHEN/THEN scenarios | EARS: WHEN [condition] THE SYSTEM SHALL [behavior] | Given/When/Then story criteria; numbered FR-1 PRD lines |
| Coding agents | 41 named integrations plus a generic option | 30+ AI tools, per its README | Kiro's own agents | 48 tools in the installer list |
| Built-in checks | Read-only analyze; converge appends unmet work as tasks | openspec validate for structure; advisory /opsx:verify | Property-based tests from requirements (IDE only); hooks | Build step checks criteria; QA skill generates and runs tests |
| Install | uv tool install specify-cli (Python 3.11+) | Install script, Homebrew, or npm with Node.js 22.12+ | IDE download; CLI install script | npx bmad-method install (Node.js 20.12+) |
| License | MIT | MIT | AWS Content, not open source | MIT |
| Latest release | v1.3.0, 3 Nov 2026 | v1.16.0, 30 Oct 2026 | Specs docs updated 15 Oct 2026 | v6.12.4, 14 Oct 2026; V7 in preview |
| Fits | New features run step by step in your own agent | Ongoing changes to an existing codebase | Teams moving into one integrated tool | Larger efforts that need product and architecture planning |
Every cell comes from the tool's own repository or documentation, read on 5 November 2026, and each section below links the file behind it. Star counts were read from the GitHub REST API, which returned this:
# GitHub REST API, 2026-11-05 14:52 UTC
github/spec-kit 147701 MIT
Fission-AI/OpenSpec 74019 MIT
bmad-code-org/BMAD-METHOD 55029 NOASSERTION
kirodotdev/Kiro 4446 none- BMAD's license - GitHub's API reports NOASSERTION; the LICENSE file is the MIT text with an added contributor attribution paragraph, and the README states MIT.
- Kiro's stars - kirodotdev/Kiro is Kiro's public issue and feedback tracker, and its README says the product source code is not hosted there, so its star count is not comparable with the other three.
What Is GitHub Spec Kit?
GitHub Spec Kit is an MIT-licensed toolkit that gives a coding agent a fixed spec-driven process: its specify CLI sets up the project, and commands you run in the agent's chat take each feature from a spec to working code. Release v1.3.0 shipped on 3 November 2026, per the Spec Kit repository, which also bundles bug-fixing and idea-assessment processes as extensions.
Spec Kit needs Python 3.11 or newer and uv, and its integrations page lists 41 named coding agents, including Claude Code, GitHub Copilot, Codex CLI, Cursor and Gemini CLI, plus a generic option for the rest. For Claude Code, the skills it installs are invoked with a hyphen, as /speckit-specify, per the Claude Code integration guide.
uv tool install specify-cli
specify init my-project --integration claude- /speckit-constitution - sets project principles, once per project.
- /speckit-specify - writes spec.md with user stories, acceptance scenarios, functional requirements and success criteria.
- /speckit-plan - writes plan.md plus research.md, data-model.md, quickstart.md and the contracts/ folder.
- /speckit-tasks - breaks the plan into tasks.md.
- /speckit-implement - executes the tasks.
- /speckit-converge - inspects the code against spec.md, plan.md and tasks.md and appends any unmet work as new tasks. The README says to repeat implement and converge until convergence reports Converged.
Three optional quality gates sit between those steps: /speckit-clarify before planning, /speckit-checklist after it, and /speckit-analyze, a read-only consistency check across spec.md, plan.md and tasks.md. Each feature ends up with this folder:
specs/001-coupon-code/
├── spec.md # /speckit-specify
├── checklists/
│ └── requirements.md # /speckit-specify
├── plan.md # /speckit-plan
├── research.md # /speckit-plan
├── data-model.md # /speckit-plan
├── quickstart.md # /speckit-plan
├── contracts/ # /speckit-plan
└── tasks.md # /speckit-tasksSpec Kit v1.3.0 feature folder, per the official plan and specify templates, November 2026; the folder name is an example. The spec template asks for prioritized user stories, each with Given/When/Then acceptance scenarios, then functional requirements numbered FR-001 and measurable outcomes numbered SC-001. A filled-in excerpt looks like this:
### User Story 1 - Apply a coupon code (Priority: P1)
**Acceptance Scenarios**:
1. **Given** a cart totalling 50.00, **When** the
user applies SAVE10, **Then** the total shows 45.00
### Functional Requirements
- **FR-001**: System MUST reject expired codes
- **FR-002**: Users MUST be able to remove a code
### Measurable Outcomes
- **SC-001**: Shoppers apply a code in under 10 secondsSpec Kit's checks run inside the agent, and /speckit-converge changes no application code. Test tasks appear in tasks.md only when the feature specification asks for tests, per the tasks template, which then has them written before the code they cover.
What Is OpenSpec, and What Does openspec validate Check?
OpenSpec is an MIT-licensed spec framework for AI coding assistants that plans work one change at a time. Each change gets a folder with a proposal, a design, a task list and spec deltas, and archiving the change merges those deltas into the main specs. The OpenSpec README says it works with 30+ AI tools, and release v1.16.0 shipped on 30 October 2026.
The README's quick start uses an install script on macOS and Linux and a PowerShell script on Windows; npm (Node.js 22.12.0 or higher) and Homebrew also work. Then run openspec init in the project, which sets up the slash commands for the tools you pick:
npm install -g @fission-ai/openspec@latest
openspec init- Default profile -
/opsx:explore,/opsx:propose,/opsx:applyand/opsx:archive, plus update and sync. - Expanded profile - adds
/opsx:new,/opsx:continue,/opsx:ff,/opsx:verify,/opsx:bulk-archiveand/opsx:onboard; select it withopenspec config profile. - Spelling per tool -
/opsx:proposeis the canonical name, and Cursor and GitHub Copilot spell it/opsx-propose;openspec initprints the right form.
openspec/
├── specs/ # current behavior
│ └── cart/
│ └── spec.md
└── changes/
└── add-coupon-codes/
├── proposal.md # why and what
├── design.md # how
├── tasks.md # implementation checklist
├── .openspec.yaml # optional metadata
└── specs/
└── cart/
└── spec.md # delta specOpenSpec v1.16.0 layout, per its concepts doc, November 2026; the change and capability names are examples. A delta spec states what the change adds, modifies or removes, with SHALL or MUST in each requirement and WHEN/THEN scenarios under it:
## ADDED Requirements
### Requirement: Coupon codes
The cart SHALL apply a valid coupon code to the
order total.
#### Scenario: Valid code
- **WHEN** the user applies SAVE10 to a 50.00 cart
- **THEN** the total shows 45.00openspec validate is a structure check, and the OpenSpec CLI reference lists what it covers. It exits 1 on a validation failure, so it can fail a CI job:
openspec validate --all --strict --json| openspec validate checks | It leaves to you |
|---|---|
| Structural issues in changes and specs | Whether the running app behaves as a scenario says |
| MODIFIED requirements against the main specs they would replace | Whether a THEN clause names an outcome a test can observe |
| That a change has spec deltas, unless .openspec.yaml sets skip_specs: true | Whether any test exists or passes |
| With the archived option: every archived change has all tasks.md boxes ticked | Whether the requirement is the right one for the product |
| With the strict option: warnings, such as a requirement without SHALL or MUST, fail the run | Anything outside the openspec/ folder |
/opsx:verify goes further than validate and stays advisory. Its verify skill has the agent search the codebase for each requirement, check whether tests cover each scenario and report on completeness, correctness and coherence; its instructions never run the test suite, and the command does not block archiving.
What Is Kiro, and How Does It Write Specs?
Kiro is Amazon's agentic development tool, available as an IDE, a CLI and a web app, with specs built in. Every spec generates requirements.md (or bugfix.md for a bug), design.md and tasks.md, per Kiro's specs documentation, updated 15 October 2026, whose capability table also shows which spec features run where:
- Feature Specs - run requirements-first or design-first, in the IDE, CLI and Web.
- Bugfix Specs - replace requirements.md with bugfix.md, which records current, expected and unchanged behavior so the fix keeps working code intact.
- Quick Spec - generates all three files in one pass without approval gates, for well-understood features.
- Analyze Requirements - looks for inconsistencies, ambiguities and gaps in the requirements before design, in the IDE and CLI.
- Correctness - property-based tests derived from the requirements, listed for the IDE only.
Kiro writes feature requirements in EARS notation, as WHEN [condition] THE SYSTEM SHALL [behavior] statements. This example comes from Kiro's requirements-first guide:
WHEN a user submits an email that already exists
THE SYSTEM SHALL display "Email already registered" error.kiro/
└── specs/
├── shopping-cart/
│ ├── requirements.md # or bugfix.md
│ ├── design.md
│ └── tasks.md
└── user-authentication/Kiro spec layout, per Kiro's specs and best-practices docs, November 2026: one folder per feature, and the folder names are examples.
Kiro's correctness feature extracts properties from the requirements and generates hundreds or thousands of random test cases when you choose to run them. The property tests are optional by default, and the correctness docs say the method provides evidence of correctness, not a proof.
Kiro hooks can also run a shell command, such as a linter or a test suite, when the agent modifies a file, invokes a tool or completes a task, per the Kiro hooks docs. Hook configurations are JSON files in .kiro/hooks/.
Kiro is the one tool here that is not open source: its license page says the Kiro IDE and Kiro CLI are licensed as AWS Content under the AWS Customer Agreement. Spec-mode prompts, spec refinement and task execution all use credits, and the Kiro pricing page lists a free plan with 50 credits a month.
What Is the BMAD Method?
The BMAD Method (Breakthrough Method for Agile AI Driven Development) is an MIT-licensed method that adds named agents and workflow skills to the AI coding tool you already use. It sizes planning to the work: small changes go straight to build, and complex work gets more planning, up to a brief, a PRD and an architecture. The stable line is v6.12.4, released 14 October 2026, while the BMAD-METHOD README on the main branch describes a V7 preview with a release expected in mid-November 2026.
This section describes v6.12.4, the version npx bmad-method install sets up. The docs site at docs.bmad-method.org already documents the V7 preview, whose file names and agent list differ, so check which line a tutorial covers before you copy its paths.
npx bmad-method installThe installer needs Node.js 20.12 or newer, and skills such as bmad-build also need Python 3.10 and uv. The v6.12.4 installer tool list offers 48 AI coding tools and marks Claude Code, Codex, Cursor and GitHub Copilot as preferred. The skills and agents reference lists five named agents:
- Analyst (Mary) - research, product briefs and project context.
- Product Manager (John) - PRDs, epics and stories, and implementation readiness.
- Architect (Winston) - the architecture spine.
- Developer (Amelia) - build, QA test generation, code review and sprint planning.
- UX Designer (Sally) - UX design.
_bmad-output/
├── planning-artifacts/
│ ├── briefs/brief-<project>-<date>/brief.md
│ ├── prds/prd-<project>-<date>/prd.md
│ ├── architecture/architecture-<project>-<date>/
│ │ └── ARCHITECTURE-SPINE.md
│ └── epics.md
└── specs/
└── spec-<slug>/
├── SPEC.md
└── stories.yaml # optionalBMAD v6.12.4 default output paths, from the module and skill configuration files in the official repository, November 2026; you choose the folders at install. Per the planning path guide, a well-defined intent goes to bmad-spec, which writes SPEC.md and, on request, stories.yaml. Stories in epics.md state acceptance criteria as Given/When/Then:
### Story 1.1: Apply a coupon code
As a shopper,
I want to apply a coupon code at checkout,
So that I pay the discounted price.
**Acceptance Criteria:**
**Given** a cart totalling 50.00
**When** I apply SAVE10
**Then** the total shows 45.00BMAD's build step has the agent confirm that every task is done and every acceptance criterion is satisfied, per its implement step. After that, the Developer's QA code runs bmad-qa-generate-e2e-tests, which generates API and end-to-end tests from the existing code, runs them and fixes failures, per the BMAD testing guide. The separate Test Architect module adds test design, traceability and release gates.
Spec Kit vs OpenSpec: What's the Difference?
Spec Kit and OpenSpec differ mainly in the unit of work. Spec Kit plans a whole feature in a numbered folder, while OpenSpec plans one change at a time as a delta against specs that describe how the system behaves today. OpenSpec's concepts doc calls it brownfield-first, and archiving moves each finished change to changes/archive/ with a date prefix.
| Aspect | Spec Kit | OpenSpec |
|---|---|---|
| Starts from | A feature description | A proposed change to existing specs |
| Requirement format | Given/When/Then scenarios with FR and SC IDs | SHALL or MUST requirements, WHEN/THEN scenarios, ADDED/MODIFIED/REMOVED sections |
| End of the loop | Repeat implement and converge until it reports Converged | Archive merges the deltas into openspec/specs/ |
| Consistency check | /speckit-analyze, run by the agent | openspec validate, a CLI command that can fail CI |
| CLI runtime | Python 3.11+ with uv | Standalone installer, or Node.js 22.12+ for npm |
| Agent coverage | 41 named integrations | 30+ tools, per its README |
Pick Spec Kit when most of your work is new features and you want each one to pass through the same sequence. Pick OpenSpec when the codebase already has behavior worth preserving and you want every change reviewed as a small diff against it.
BMAD vs Spec Kit: What's the Difference?
BMAD and Spec Kit differ in scope. Spec Kit is a lean pipeline for one feature at a time, driven by commands in your coding agent, while BMAD is a fuller method whose agents and workflows can plan a whole product before stories are built.
- Roles - BMAD v6.12.4 installs five named agents, from Analyst to Developer; Spec Kit has commands and no personas.
- Planning depth - BMAD can produce a brief, a PRD, an architecture spine and epics before a story is built; Spec Kit starts from one feature description.
- Requirement style - both use Given/When/Then; BMAD's PRD adds numbered FR-1 requirements with testable consequences, and Spec Kit uses FR-001 and SC-001.
- Testing - BMAD ships a QA test-generation skill and an optional Test Architect module; Spec Kit adds test tasks only when the spec asks for tests.
- Runtime - Spec Kit's CLI needs Python 3.11+ and uv; BMAD's installer needs Node.js 20.12+, with Python and uv for some skills.
Spec Kit is quicker to adopt for a single developer or a single feature. BMAD gives more structure when several people must agree on what the product is, which is when its planning guide says a PRD is worth writing.
Kiro vs Spec Kit: What's the Difference?
Kiro builds specs into its own tool, while Spec Kit adds them to whichever coding agent you already use. The two also meet: Spec Kit's Kiro CLI integration installs Spec Kit's commands as Markdown prompts under .kiro/prompts/, so a team can keep Kiro CLI and use Spec Kit's spec format.
- Requirement format - Kiro writes EARS statements in requirements.md; Spec Kit writes user stories with Given/When/Then scenarios in spec.md.
- Tests from the spec - Kiro's IDE can generate property-based tests from requirements; Spec Kit writes test tasks when the spec asks for them, and the agent implements those tests.
- Where it runs - Kiro runs its own agents in its IDE, CLI and web app; Spec Kit runs in 41 named agents, Kiro CLI among them.
- License and cost - Kiro is AWS Content and bills in credits, with a free plan of 50 credits a month; Spec Kit is MIT-licensed and costs nothing beyond your coding agent's own usage.
Choose Kiro if your team will move to its IDE and wants specs, hooks and property tests in one place. Choose Spec Kit if each developer keeps their own agent.
How Do You Add a Testing Step to Any Spec-Driven Development Tool?
Add a testing step by designing tests from the spec's acceptance criteria, running them against the built app, and failing the build when a criterion has no passing result. All four tools write their requirements as Markdown, so a test tool that reads Markdown can start from the same file the coding agent built from.
Most of the post-coding checks these tools document run inside the same agent workflow that wrote the code: Spec Kit's converge reads the code against the artifacts, OpenSpec's verify writes an advisory report, BMAD's build step confirms its own criteria, and Kiro runs its property tests during task execution. Tests designed from code can inherit its bugs: in an experiment TestMu AI published in October 2026, 22 of 60 unit test suites that two Gemini models wrote from the code passed on a seeded bug, while none of the 60 suites written from each function's documented contract did, as described in why AI-generated tests pass when your app is broken.
- Point at the requirements file - spec.md in a Spec Kit feature folder, a delta spec in an OpenSpec change, Kiro's requirements.md, or a BMAD SPEC.md or story.
- Give every criterion an expected result - a criterion with no observable outcome can neither pass nor fail; these acceptance criteria examples show the difference.
- Design one test per scenario - tag each test with the IDs of the criteria it proves, so a failure names the requirement. The split between a criterion and the test that proves it is explained in acceptance criteria vs acceptance tests.
- Run against the built app in CI - fail the job when a criterion is unproven, and keep each run's evidence with the build, which also keeps your requirements traceability matrix current.
TestMu AI's Kane CLI runs that loop from the terminal. It is free to install, every run uses credits from your TestMu AI account, and you can claim 10,000 credits, valid for 30 days, on the Kane CLI page. Its requirement sources docs list Markdown among the accepted file types, and the loop for a Spec Kit spec looks like this:
npm install -g @testmuai/kane-cli
kane-cli context ingest specs/001-coupon-code/spec.md
kane-cli context review # approve use-cases
kane-cli design tests --use-case uc-1
kane-cli context review # approve the design
kane-cli testrun run --headless --agent
kane-cli cover # proven vs still owed- Cited use-cases - after ingest, an agent proposes use-cases that cite the exact lines of the spec it read, and nothing becomes trusted until you approve it.
- Missing expected results - when the spec never states the expected result for a promise, kane-cli design tests records a missing-expected-result gap with a recommended default instead of inventing one, per the Kane CLI test design docs.
- Tagged tests - design writes one runnable test per scenario, and each assert step lists the acceptance criteria it verifies.
- Coverage from evidence - every run seals an evidence pack, and kane-cli cover reports what a run proved next to what the design still owes.
- Formats tested so far - TestMu AI has published a full run on a Spec Kit spec.md in its Spec Kit with Claude Code walkthrough. Kiro, OpenSpec and BMAD files are Markdown and fall under the accepted type, but how well extraction handles their layouts is not documented yet.
Note: Point Kane CLI from TestMu AI at the spec your coding agent just built from, and see which acceptance criteria a real browser run proves before you merge. Sign up for free.
Questions to Settle Before You Choose a Spec-Driven Development Tool
Answer these in order before a team rollout, because the release pace is high: Spec Kit shipped v1.2.6 on 2 November 2026 and v1.3.0 a day later, and the BMAD README expects V7 in mid-November 2026.
- Unit of work - decide whether you plan whole features (Spec Kit), changes to existing behavior (OpenSpec), or products with several epics (BMAD).
- Coding agent - decide whether each developer keeps their own agent or the team moves to Kiro's IDE, CLI or web app.
- Requirement format - pick the format your reviewers and testers read most easily: Given/When/Then, SHALL with WHEN/THEN, or EARS.
- License and cost - Spec Kit, OpenSpec and BMAD are MIT-licensed; Kiro bills in credits under the AWS Customer Agreement.
- Verification - decide which check fails the build: a structure check, the agent's own review, or tests run against the built app.
Start with one feature: write its spec in the tool you pick, ingest that same file into Kane CLI, and merge only when the criteria you care about show as proven. The Kane CLI coverage docs explain how proven and owed criteria are counted.
Author
Sirajuddin Khan is Vice President of Product Management at TestMu AI, where he drives the company's agentic AI product strategy, building a suite of autonomous agents that includes Agentic Browsers and Agentic Visual Testing and shifting the unit of work from test execution to autonomous outcomes. One of the company's earliest product leaders, he has owned the roadmap for the high-performance execution cloud and grew the cross-browser testing products from early adoption to market leadership. He brings over a decade of experience across SaaS, B2B, and eCommerce, with earlier product roles at Wydr and ShopClues, where his catalog and search work cut delivery SLAs and lifted seller activity. Sirajuddin holds an MBA in Information Technology from Sikkim Manipal University and a B.Tech in Computer Science Engineering from Maharshi Dayanand University.
Reviewer
Mayank Bhola is Co-Founder and Head of Products at TestMu AI, where he leads the entire product portfolio across KaneAI, Kane CLI, HyperExecute, SmartUI, the Real Device Cloud, Accessibility, and other software testing product lines. As an early Lead Architect he designed and built the company's flagship Tunnel technology from scratch, created the React-based automation platform, and architected the data-intensive pipelines and FAAS services that scale it. He brings more than 10 years of experience in software development and product engineering, with earlier roles as Head of Technology at Juggernaut Books and Senior Software Engineer at PressPlay TV and Zomato. Mayank holds a B.Tech in Computer Engineering from JIIT Noida.
Spec-Driven Development Tools FAQs
Did you find this page helpful?
More Related Blogs
TestMu AI forEnterprise
Get access to solutions built on Enterprise
grade security, privacy, & compliance
- Advanced access controls
- Advanced data retention rules
- Advanced Local Testing
- Premium Support options
- Early access to beta features
- Private Slack Channel
- Unlimited Manual Accessibility DevTools Tests





