Playwright CLI vs Playwright MCP vs Kane CLI: What the Docs Say, What the Data Shows

The short answer: Playwright CLI vs Playwright MCP vs Kane CLI

What is Playwright CLI?

Playwright CLI (playwright-cli) is Microsoft's command-line tool for driving a browser from the shell, built with coding agents in mind. Each command does one thing (open a page, click, type, take a snapshot) and prints a short result. The CLI writes page snapshots to a file, and the output points to that file instead of printing the whole page. The agent reads the file only when it needs to.

npm install -g @playwright/cli@latest
playwright-cli install --skills   # teaches Claude Code, Copilot and others how to use it
playwright-cli --help

Sessions live in memory by default, so cookies survive between commands but disappear when the browser closes. Add --persistent to keep the profile on disk, or -s=name to run several named sessions side by side.

What is Playwright MCP?

Playwright MCP is Microsoft's MCP server that gives an agent a browser as a set of tools. Each action returns an accessibility snapshot of the page straight into the model's context, so the agent always "sees" the current state. That helps when the agent is exploring and gets expensive on busy pages. For setup, flags and safety, see our Playwright MCP guide.

What is Kane CLI?

Kane CLI is TestMu AI's testing agent for the terminal. You give it one goal in plain English. It drives a real Chrome browser itself, checks the outcome against the criteria in your goal, and returns one result: pass, fail, setup error or timeout. Every run produces an .evidence pack with steps, screenshots and logs. Flows you save replay on later runs and re-author only the steps your UI changed.

npm install -g @testmuai/kane-cli
npx @testmuai/kane-cli-skill   # Claude Code, Codex CLI and Gemini CLI

What's the difference between MCP vs CLI?

The difference between MCP vs CLI is where the page data goes. With MCP, the snapshot goes into the model's context after every action. With CLI, the snapshot goes into a file, and the agent decides whether to read it.

Model APIs are stateless, so every turn re-sends the whole conversation, including every snapshot taken so far. You pay for a big snapshot in context again on every later turn, which is the case for CLI.

Is Playwright CLI really cheaper than Playwright MCP?

Per command, yes. Per task, it depends on how many turns the agent takes.

Microsoft's README says Playwright CLI with skills is more token-efficient for coding agents because it keeps large tool schemas and accessibility trees out of context. Slack Engineering then ran the same complex flow through both, with the same model (Slack Engineering, June 2026):

Across their full experiment, Playwright CLI also failed more often (about 12 to 20% versus 0 to 12% for MCP), mostly on login, timing and session handling rather than reasoning.

Both claims hold. Each CLI command is cheap, but one browser action through the CLI often becomes several commands (act, wait, snapshot, read the file, find the element), and each extra turn re-sends the whole conversation. CLI wins on a short task, and on a long one the turn count can eat the savings. So measure turns per task, not tokens per call.

What does the same task look like in each tool?

Task: in a notes app, create a note called "Q4 plan", tag it "work", search for it and confirm it shows up.

Playwright CLI. The agent runs a sequence like this (the e references come from the snapshot and will differ on your page):

playwright-cli open https://notes.staging.app
playwright-cli snapshot
playwright-cli click e14          # "New note"
playwright-cli fill e22 "Q4 plan"
playwright-cli click e31          # tag picker
playwright-cli click e38          # "work"
playwright-cli fill e9 "Q4 plan"  # search box
playwright-cli snapshot           # agent reads the file to check results

The agent then reads the last snapshot and decides for itself whether the note appeared.

Playwright MCP. The same steps happen as tool calls (browser_navigate, browser_click, browser_type and so on), and every call returns the page snapshot into context. The agent again decides whether it worked.

Kane CLI. One command runs the flow, and this time the check decides whether it worked:

kane-cli run "Create a note called 'Q4 plan', tag it 'work', search for 'Q4 plan', \
  and verify the note appears in the results with the 'work' tag" \
  --url https://notes.staging.app --agent --headless \
  | tail -1 | jq '{status, one_liner, test_url}'

Your coding agent gets one line back. Exit code 0 means pass, 1 fail, 2 setup error, 3 timeout.

Playwright CLI vs Playwright MCP vs Kane CLI: full comparison

Which should run in CI?

Neither Playwright CLI nor Playwright MCP is built to be a CI gate on its own, because the agent's judgment decides the result. In CI you want a fixed pass/fail rule, a time limit and a predictable cost. Use Playwright Test for code you maintain, or Kane CLI for flows written as intent:

- run: npm install -g @testmuai/kane-cli
- name: Verify critical flows
  env:
    LT_USERNAME: ${{ secrets.LT_USERNAME }}
    LT_ACCESS_KEY: ${{ secrets.LT_ACCESS_KEY }}
  run: |
    kane-cli testrun run --tags smoke --parallel 4 --on-failure fail-fast \
      --headless --username "$LT_USERNAME" --access-key "$LT_ACCESS_KEY"

How do you use all three without paying twice?

Give your agent clear jobs for each tool. Put rules like these in your CLAUDE.md or AGENTS.md:

## Browser tools
- To look at the app while coding (inspect a page, read console errors,
  try a click): use playwright-cli.
- For long exploratory sessions where you need to keep reasoning over the
  page: use Playwright MCP.
- Before saying a UI change is done: run
  kane-cli run "<goal and expected result>" --agent --headless
  and only report done on exit code 0. Include the test_url.
- Never decide pass/fail from your own reading of a snapshot.

The last rule separates the agent that writes the code from the check that proves it works.

Install Kane CLI