How a coding agent verifies a change with Kane CLI

When a coding agent should verify a change

Run this check every time you change something an end user can see or click, such as a form, a page, a checkout step, a sign-in flow or a layout fix. Run it before you tell the developer the work is done.

Steps to verify a change in a real browser

1. 1

   Start the app or open its preview URL

   Run the dev server (for example npm run dev) or take the preview URL from the deploy. Kane CLI needs a URL it can open, either at the start of the objective or passed with --url; a non-interactive run with no start URL fails instead of guessing. Before the first run, check that kane-cli whoami shows a signed-in user and that Chrome is installed, because the npm package does not install it.

2. 2

   Write one objective that ends with a verify check

   Start with the URL, describe the actions as goals, and end with a check of what should be on screen when the change works:

   textCopy

   Go to http://localhost:3000/products, filter by size M, then verify every product card shows size M

3. 3

   Run kane-cli with --agent and --headless

   --agent prints NDJSON on stdout, one JSON object per line, and keeps the human progress view on stderr. Add --headless on any machine without a display, and --timeout <seconds> so a stuck run ends. Give your shell tool a longer timeout than that: runs take from 30 seconds to a few minutes.

   bashCopy

   kane-cli run "Go to http://localhost:3000/products, filter by size M, then verify every product card shows size M" --agent --headless --timeout 180

4. 4

   Read the run_end result on the last stdout line

   Check the exit code first: a run refused before it starts exits 2 and can end with no run_end. Otherwise the first line is stream_start, step lines have no type field and carry step, status (running, done or failed) and remark, and the last line is run_end. An example run_end line, with values that will differ on your run:

   jsonCopy

   {"type":"run_end","status":"passed","summary":"Filtered by size M and every product card shows size M","one_liner":"Size filter works","reason":"Objective completed","duration":19.8,"credits_consumed":8.4,"final_state":{},"test_url":"https://test-manager.lambdatest.com/…","session_dir":"…"}

   Fields to rely on: status, summary, one_liner, reason, duration, credits_consumed, final_state (values saved with "store … as"), test_url (the Test Manager link, present when the upload succeeded) and session_dir. Ignore fields and event types you do not recognize, because new ones can appear in any release.

5. 5

   Act on the kane-cli exit code

   Branch on the exit code: 0 passed, 1 ran and did not pass, 2 did not start, 3 timed out or was cancelled. Use run_end for the details you report.

   • 0 passed: tell the developer, with the summary and the test_url Test Manager link.
   • 1 failed: read the failing step's remark first (the line with "status":"failed"). If the app is wrong, fix the code and run the same command again. If the objective was vague or a step got stuck, rewrite the objective or tell the developer, and never weaken the check to get a pass. A failed upload also exits 1: if no step failed and test_url is missing, report that instead of changing code.
   • 2 did not start: nothing ran and no credits were used. Check three things: sign-in (kane-cli whoami), that Chrome is installed, and the start URL. If the developer is not signed in, run kane-cli login --oauth for them (it opens their browser and waits), or ask them to run kane-cli login in their own terminal.
   • 3 timeout or cancelled: raise --timeout or split the objective into shorter runs.