Synthetic Monitoring with Kane CLI: Check Live User Flows

Defining Synthetic Monitoring

By simulating critical business interactions like authentication, multi-tenant searches, cart additions, and payment processing, synthetic monitoring flags environment drift, certificate expirations, and third-party API degradation. It catches these problems before a real user ever encounters them.

When the scripted journey is a full business transaction rather than a single page load, the practice is called synthetic transaction monitoring. An uptime ping confirms that a URL answers. A transaction check confirms that a signed-in user can still search, add to cart, and reach the order confirmation page. For the broader testing discipline behind it, see this guide to synthetic testing.

Synthetic Monitoring vs Real User Monitoring

The two answer different questions, and most teams run both.

• Real user monitoring (RUM) - records what actual visitors experience: page speed, errors, and drop-offs across real devices and networks. It is reactive, so a broken checkout shows up only after customers fail to pay, and at 3 a.m. there may be too little traffic for it to show up at all.
• Synthetic monitoring - runs the same scripted journey on a schedule whether or not anyone is on the site. It is proactive and repeatable, so a failure points to a specific step in a specific flow, with no customer impact needed to detect it.

Use synthetic checks to know when a critical flow breaks, and RUM to learn how real users experience the flows that work.

Pre-Merge CI Gating vs Scheduled Production Monitoring

Both run browser checks, but they serve different parts of the lifecycle. The distinction is about when each check runs and what kind of failure it is positioned to catch.

• CI gating - runs on pull requests to catch logical regressions before merge. See how to gate a pull request with Kane CLI.
• Synthetic monitoring - runs continuously on a cron schedule against the live environment to catch infrastructure drift, CDN routing failures, database locks, and third-party API issues a pre-merge check can never see.

With Kane CLI you do not maintain two frameworks. The same plain-English objective that gates a PR drops into a GitHub Actions schedule, so the check that runs in CI/CD also runs your production monitor. For the one-off check right after each release, see smoke testing command line.

Note: Put your critical flows on a schedule and alert on the first failure. Start free. Try Kane CLI

Choosing Which Flows to Monitor and How Often

A monitor that checks everything gets slow, expensive, and noisy. Start with three to five journeys where a silent failure costs money or locks users out:

• Sign-in - if users cannot log in, every other flow is broken for them too.
• Search or browse - the entry point to anything users buy or read.
• Checkout or sign-up - the step where revenue or a new account is created.
• A third-party dependency - a payment form, a map, or an embedded widget you do not control and cannot test before merge.

On a Shopify store, monitor checkout up to the payment step and cover the payment itself with a Shopify test order on a development store.

For frequency, the GitHub Actions schedule documentation sets the shortest interval at once every 5 minutes. Every 15 minutes keeps detection time short without running a browser around the clock. The same page warns that scheduled runs can be delayed at busy times, especially at the start of every hour, and that some queued runs may be dropped, so schedule your checks at off-peak minutes such as :07, :22, :37, and :52.

Bundling Critical Business Flows Into a Central Script

Bundle your highest-value paths into one script so a failure on any journey fails the whole run. Save it at monitoring/synthetic-monitor.sh. The script reads the Kane CLI exit code for each flow, because a monitor that pages someone for a flaky runner gets muted within a week.

For a quick visual walkthrough of this concept, watch the video below from TestMu AI.

Subscribe to the TestMu AI YouTube channel for more tutorials on testing and automation.

#!/usr/bin/env bash
set -uo pipefail

FAILED=0   # a flow's assertion failed (exit 1): the flow is broken
ERRORED=0  # auth, Chrome or timeout problem that survived a retry (exit 2 or 3)

run_flow () {
  local name="$1"
  local objective="$2"
  local code
  echo "::group::${name}"
  for attempt in 1 2; do
    kane-cli run "$objective" \
      --name "monitor-${name}" \
      --headless --agent --timeout 300 \
      --username "$LT_USERNAME" --access-key "$LT_ACCESS_KEY"
    code=$?
    # Retry once, only for environment errors (2) and timeouts (3)
    if [ "$attempt" -eq 1 ] && { [ "$code" -eq 2 ] || [ "$code" -eq 3 ]; }; then
      echo "${name}: exit ${code}, retrying once"
      continue
    fi
    break
  done
  case "$code" in
    0) echo "PASSED: ${name}" ;;
    1) echo "FAILED: ${name}"; FAILED=1 ;;
    *) echo "ERROR: ${name} (exit ${code})"; ERRORED=1 ;;
  esac
  echo "::endgroup::"
}

run_flow "login"    "go to https://myapp.com, log in as '{{user}}', assert the dashboard shows 'Welcome'"
run_flow "search"   "go to https://myapp.com, search for 'headphones', assert results appear"
run_flow "checkout" "go to https://myapp.com, complete checkout with a test card, assert the order confirmation page shows"

if [ "$FAILED" -eq 1 ]; then RESULT=failed
elif [ "$ERRORED" -eq 1 ]; then RESULT=error
else RESULT=passed; fi
echo "result=${RESULT}" >> "${GITHUB_OUTPUT:-/dev/null}"

[ "$RESULT" = passed ]

The --name flag matters here. The Kane CLI CI/CD documentation explains that a named run copies its sealed evidence pack into .testmuai/evidence/ in the workspace, while an unnamed run keeps it outside the workspace, where the workflow cannot archive it. The same page lists the exit codes the script branches on: 0 passed, 1 failed, 2 error, and 3 timeout or cancellation.

When asserting against live production, use dedicated isolated test accounts and read-only queries where possible. This avoids polluting analytics or triggering real charges.

Setting Up the Scheduled GitHub Actions Workflow

The workflow below adds the monitoring-specific parts: an off-peak schedule, a concurrency guard so a slow run never overlaps the next one, an evidence upload, and an alert that says whether a flow broke or the check itself could not run. For the base setup, including storing LT_USERNAME and LT_ACCESS_KEY as repository secrets, follow the step-by-step guide to connect Kane CLI to GitHub Actions.

name: Synthetic Monitoring
on:
  schedule:
    # Every 15 minutes, away from the top of the hour
    - cron: '7,22,37,52 * * * *'
  workflow_dispatch:

concurrency:
  group: synthetic-monitoring
  cancel-in-progress: false

jobs:
  monitor:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - name: Install Chrome
        uses: browser-actions/setup-chrome@v1
      - name: Install Kane CLI
        run: npm install -g @testmuai/kane-cli
      - name: Run synthetic checks
        id: monitor
        env:
          LT_USERNAME: ${{ secrets.LT_USERNAME }}
          LT_ACCESS_KEY: ${{ secrets.LT_ACCESS_KEY }}
        run: |
          chmod +x ./monitoring/synthetic-monitor.sh
          ./monitoring/synthetic-monitor.sh
      - name: Keep the evidence packs
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: synthetic-monitor-evidence
          path: .testmuai/evidence/
          retention-days: 30
      - name: Alert Slack on failure
        if: failure()
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
          RESULT: ${{ steps.monitor.outputs.result }}
          RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
        run: |
          if [ "$RESULT" = failed ]; then
            TEXT="Synthetic monitor: a user flow is broken. Evidence and ShareLink: $RUN_URL"
          else
            TEXT="Synthetic monitor: the check could not run (auth, Chrome or timeout). Details: $RUN_URL"
          fi
          curl -s -X POST -H 'Content-type: application/json' \
            --data "$(jq -n --arg t "$TEXT" '{text: $t}')" \
            "$SLACK_WEBHOOK_URL"

Three scheduling rules from the GitHub documentation catch teams out. Scheduled workflows run only on the default branch, so the monitor starts after the file is merged there. The cron runs in UTC unless you add a timezone key. In a public repository, scheduled workflows are disabled automatically after 60 days with no repository activity, which can switch a monitor off without any alert.

Proactive Failure Alerting and Triage

When a flow breaks, the script exits non-zero and fails the step. The two if: failure() steps then archive the evidence and post to the on-call Slack channel, with a message that separates a broken flow (exit 1) from a check that could not run (exit 2 or 3 after a retry). On-call engineers can treat the first as an incident and the second as a monitoring problem.

Because Kane CLI captures video records and step-by-step logs for every run, the Actions log shows a secure ShareLink. On-call engineers click it to watch the failure replay, reducing MTTR. The Test Manager integration docs note that share links expire after seven days, so for a postmortem written later, open the evidence pack from the workflow artifact, which the workflow above keeps for 30 days.

A missing run is not a passing run. Since GitHub can delay or drop a scheduled run, check the workflow's run history after an incident, and if the monitor guards revenue, have an external heartbeat service expect a ping from each successful run and alert when the pings stop.

Note: Want the agent mode and credentials setup behind this monitor? Read the Kane CLI docs. Read the docs

The same objective that runs every 15 minutes in production also runs before merge, the way Kane CLI handles a UI check before a PR ever lands.

Testing from the terminal? Kane CLI by TestMu AI lets SDETs and developers plan, write, run and debug tests from the command line.

To run your first test, start free, or

book a demo

for a full walkthrough of the product.