How to Run Kane CLI in CI/CD

​

Common Patterns

• Always pass --headless. CI runners have no display.
• Always set --timeout <seconds>, so a hung run cannot block the pipeline.
• Authenticate with --username and --access-key from CI secrets. Do not call kane-cli login in CI: that flow opens a browser for OAuth and will not work on a runner.
• Provide a start URL. A CI run cannot prompt for input. If the objective names no site, no --url flag is passed, and no default_url is configured, the run fails fast instead of waiting. The simplest fix is to start the objective with the site (“Go to https://… and …”) or pass --url <url>. To start from the browser’s current page on purpose, add --allow-missing-url. See Default start URL.
• Load test data with --variables-file <path>. Commit the file to your repo without secret values, or generate it before the step, filling values from CI secrets. A {{name}} with no value does not change the exit code on its own. Kane CLI warns before the browser starts and then types the name as written. If a step fails because of that literal text, the job fails as usual. The warning is a warning line on stdout with code: "unresolved_variables". If a missing value should block the pipeline, fail the step when that line appears.
• Project and folder are optional. To file uploads under a specific Test Manager project or folder, set them in advance with kane-cli config project <id> and kane-cli config folder <id>. Find the IDs with kane-cli projects list and kane-cli folders list. If you skip this, Kane CLI picks a default project and folder on the first run and reports which ones it chose. See Test Manager Integration.
• Check the exit code. In short: 0 passed, 1 failed, 2 error, 3 timeout or cancellation. The full mapping is in Exit Codes.
• Archive the evidence pack. Name a kane-cli run with --name <name>, and Kane CLI copies its sealed evidence pack into .testmuai/evidence/ in the workspace. testmd run and testrun run always write their pack there. Archive that folder as a CI artifact. Without --name, a single run keeps its pack only in its session folder under ~/.testmuai/kaneai/sessions/, outside the workspace.
• Run whole suites with one command. If your repo has committed _test.md tests, prefer one testrun invocation over a shell loop:
  kane-cli testrun run --tags smoke --parallel 4 --headless --on-failure fail-fast
  
  The suite runs as one execution, with one exit code (0 all passed, 1 any failure, 2 invalid plan, 3 cancelled) and one sealed evidence pack. Archive .testmuai/evidence/*.evidence as a CI artifact so anyone can open it in the viewer. See Batch Runs (testrun).
• Run the suite on the cloud grid when your runner cannot. kane-cli testrun run … --remote turns the suite into one HyperExecute job. The grid supplies Chrome on a macOS runner, or a virtual Android emulator or iOS simulator for mobile _test.md tests, so your runner needs no Chrome, Xcode, or Android Studio. --parallel N spreads the tests across N grid runners. The recordings and the evidence pack come back to your checkout, just as if the suite had run on your machine.
  kane-cli plugin install remote-execution
  
  # a web suite on 4 grid runners
  kane-cli testrun run --match '^tests/web/' --remote --parallel 4 \
    --username "$LT_USERNAME" --access-key "$LT_ACCESS_KEY"
  
  # a mobile suite, from a Linux runner
  kane-cli testrun run --match '^tests/app/' --remote \
    --device-name "Pixel 7" --os-version 14 \
    --username "$LT_USERNAME" --access-key "$LT_ACCESS_KEY"
  
  This requires a plan with HyperExecute macOS runners. Web and mobile tests must go in separate runs. Choose devices with kane-cli devices list --target emulator|simulator --remote, and allow a timeout of about 10 minutes. See Remote Runs.

• KANE_CLI_CDP_TIMEOUT_MS: how long each launch attempt waits for Chrome to be ready. Default 30000.
• KANE_CLI_CDP_RETRIES: how many times to retry the launch after the first attempt. Default 2.

​

Authentication in CI/CD

kane-cli run "Verify checkout flow completes" \
  --url https://staging.myapp.com \
  --username "YOUR_LT_USERNAME" \
  --access-key "YOUR_LT_ACCESS_KEY" \
  --headless \
  --agent \
  --timeout 300

​

Exit Codes

​

CI/CD Checklist

• Always use --headless and --agent: no display server in CI
• Set --timeout: prevents pipeline hangs (e.g. --timeout 300)
• Set --max-steps: caps run length (e.g. --max-steps 50)
• Use --variables-file: load test data from a committed config file
• Store credentials as secrets: never hardcode in pipeline files

​

Platform Guides

# .github/workflows/kane.yml
name: Kane browser check
on: [pull_request]

permissions:
  contents: read
  pull-requests: write   # lets the action post the verdict comment

jobs:
  verify-ui:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: LambdaTest/kane-cli/integrations/github-action/kane-cli-browser-run@main
        with:
          objective: "Go to https://staging.myapp.com/pricing and verify the Pro plan card shows a monthly price"
          username: ${{ secrets.LT_USERNAME }}
          access-key: ${{ secrets.LT_ACCESS_KEY }}

# .github/workflows/browser-tests.yml
name: Browser Tests
on: [push, pull_request]

jobs:
  kane-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install Chrome
        uses: browser-actions/setup-chrome@v1

      - name: Install Kane CLI
        run: npm install -g @testmuai/kane-cli

      - name: Run browser tests
        env:
          LT_USERNAME: ${{ secrets.LT_USERNAME }}
          LT_ACCESS_KEY: ${{ secrets.LT_ACCESS_KEY }}
        run: |
          kane-cli run \
            "Search for 'wireless headphones' on Amazon and open the first result" \
            --name amazon-search \
            --headless \
            --timeout 300 \
            --username "$LT_USERNAME" \
            --access-key "$LT_ACCESS_KEY" \
            --variables-file ./tests/variables.json

      - name: Upload evidence
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: kane-evidence
          path: |
            .testmuai/evidence/
            **/output-*/Result.md
          include-hidden-files: true

# .gitlab-ci.yml
stages:
  - test

kane-cli:
  stage: test
  image: node:20
  before_script:
    - apt-get update && apt-get install -y wget gnupg
    - wget -qO- https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
    - echo "deb http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list
    - apt-get update && apt-get install -y google-chrome-stable
    - npm install -g @testmuai/kane-cli
  script:
    - |
      kane-cli run "Verify the homepage loads and the login button is visible" \
        --name homepage-check \
        --headless \
        --timeout 300 \
        --username "$LT_USERNAME" \
        --access-key "$LT_ACCESS_KEY" \
        --variables-file ./tests/variables.json
  variables:
    LT_USERNAME: $LT_USERNAME
    LT_ACCESS_KEY: $LT_ACCESS_KEY
  artifacts:
    paths:
      - .testmuai/evidence/
    when: always
    expire_in: 7 days

// Jenkinsfile
pipeline {
    agent any
    environment {
        LT_USERNAME   = credentials('lt-username')
        LT_ACCESS_KEY = credentials('lt-access-key')
    }
    stages {
        stage('Install') {
            steps {
                sh 'npm install -g @testmuai/kane-cli'
            }
        }
        stage('Run kane-cli') {
            steps {
                sh '''
                    kane-cli run "Sign in and confirm the dashboard renders" \
                        --name dashboard-check \
                        --headless \
                        --timeout 300 \
                        --username "$LT_USERNAME" \
                        --access-key "$LT_ACCESS_KEY" \
                        --variables-file ./tests/variables.json
                '''
            }
        }
    }
    post {
        always {
            archiveArtifacts artifacts: '.testmuai/evidence/*.evidence',
                             allowEmptyArchive: true
        }
    }
}

# bitbucket-pipelines.yml
pipelines:
  default:
    - step:
        name: Browser Tests
        image: node:20
        script:
          - npm install -g @testmuai/kane-cli
          - kane-cli run
              --url https://staging.myapp.com
              --name checkout-flow
              --username $LT_USERNAME
              --access-key $LT_ACCESS_KEY
              --headless
              --agent
              --timeout 300
              --max-steps 50
              "Complete the checkout flow and verify order confirmation"
        artifacts:
          - .testmuai/evidence/**

kane-cli run "Open the pricing page and verify the Pro plan is listed" \
    --headless \
    --timeout 300 \
    --username "$LT_USERNAME" \
    --access-key "$LT_ACCESS_KEY" \
    --variables-file ./tests/variables.json

kane-cli run "Open the pricing page and verify the Pro plan is listed" \
    --headless \
    --timeout 300 \
    --ws-endpoint "$LT_BROWSER_WSS" \
    --username "$LT_USERNAME" \
    --access-key "$LT_ACCESS_KEY" \
    --variables-file ./tests/variables.json

​

Running Multiple Tests

#!/bin/bash
set -e

PASS=0
FAIL=0
FAILED_TESTS=()

run_test() {
  local name="$1"
  local objective="$2"
  echo "Running: $name"
  if kane-cli run "$objective" \
      --url https://staging.myapp.com \
      --username $LT_USERNAME \
      --access-key $LT_ACCESS_KEY \
      --headless --agent --timeout 120; then
    ((PASS++))
  else
    ((FAIL++))
    FAILED_TESTS+=("$name")
  fi
}

run_test "Login" "Log in with valid credentials and verify dashboard appears"
run_test "Search" "Search for 'laptop' and verify at least one result appears"
run_test "Checkout" "Add first product to cart and complete checkout"
run_test "Settings" "Open account settings and verify profile page loads"

echo ""
echo "Results: $PASS passed, $FAIL failed"
if [[ $FAIL -gt 0 ]]; then
  echo "Failed tests: ${FAILED_TESTS[*]}"
  exit 1
fi

​

Variables in CI/CD

{
  "app_url": { "value": "https://staging.myapp.com" },
  "test_product_sku": { "value": "PROD-001" }
}

kane-cli run "Log in as {{email}} with {{password}} and verify dashboard" \
  --variables-file ./test-variables.json \
  --variables "{\"email\": {\"value\": \"$TEST_EMAIL\"}, \"password\": {\"value\": \"$TEST_PASSWORD\", \"secret\": true}}" \
  --username $LT_USERNAME \
  --access-key $LT_ACCESS_KEY \
  --headless --agent

​

A shared context store in CI

• Authenticate without a person. The method depends on the location:
  • GitHub over HTTPS: set KANE_SYNC_GIT_TOKEN from a CI secret. Use a repository-scoped personal access token with Contents read and write, or a GitHub App installation token. A workflow’s own GITHUB_TOKEN reaches only the workflow’s repository, so a separate context repository needs its own token.
  • GitHub over SSH: an SSH deploy key on the context repository works without a token. Kane CLI never answers an SSH prompt, so the runner must have the key loaded and the host key already accepted (ssh-keyscan github.com >> ~/.ssh/known_hosts).
  • For either GitHub option, repository rules must also allow the scratch reference that the connection check pushes under refs/kane/probe/. See Three kinds of location.
  • S3-compatible storage: set KANE_SYNC_S3_ACCESS_KEY_ID and KANE_SYNC_S3_SECRET_ACCESS_KEY.
  Kane CLI never writes these credentials to disk. See Context sync environment variables.
• Use --mode agent for structured output, and read the exit code:
  • 0: done.
  • 3: a person has to decide. The runner’s store is behind or has diverged, or a rebase stopped on decisions.
  • 2: a precondition failed. The location cannot be reached, keys are missing, or a rebase is still open.
• Do not answer decisions blindly. When a rebase stops on a decision, the job stops too. Save the output of kane-cli context sync status origin --json as a build artifact. It reports what is waiting, but a later job cannot replay it on its own, because answers must go to the store that holds the open rebase. The follow-up job must run on the same persisted .context/, such as a workspace or cache that survives between jobs. There, a person or an agent answers with kane-cli context sync origin --answer <id>=<choice>. keep-theirs writes nothing, but it is still a choice: the local change stays in the backup. The full contract is in Agents and CI.
• Keep .context/ out of version control. The store is never merged with git. It is shared through the location instead.

set -e   # stop at the first failing command, so every exit code below is read

# GitHub Actions step, where the secret CONTEXT_REPO_TOKEN grants Contents read and write on the context repository
export KANE_SYNC_GIT_TOKEN="$CONTEXT_REPO_TOKEN"

# first run on this runner: a store from the team location
[ -d .context ] || kane-cli context clone https://github.com/example-org/team-context.git . --mode agent

# before the run: take the team's new records, and stop the job if a person has to decide
kane-cli context pull origin --mode agent > pull.ndjson || {
  code=$?
  [ "$code" -eq 3 ] && kane-cli context sync status origin --json > sync-status.json   # the relation (behind or diverged) and any open decisions
  exit "$code"
}

# the run itself: it fails on a high-risk question and takes the recommended default for the rest; a failed or paused extraction (exit 1 or 3) ends the job here, before anything is pushed
kane-cli context extract --mode ci

# after the run: publish what landed
kane-cli context push origin --mode agent