Running Automated Tests Against Vercel Preview URLs

How Previews Actually Trigger

Vercel's environments documentation defines three defaults: Local, Preview, and Production. A Preview deployment is created when you push a commit to a non-production branch, open a pull request, or run the CLI without --prod.

One exception worth knowing before it confuses a first CI run: the very first deployment of a brand-new project is always a production deployment, regardless of which branch or flag you used.

The Two Preview URL Types

Grab the commit-specific URL from the deployment event rather than constructing the branch URL yourself; the next section covers exactly where that URL lives.

Getting the URL Into CI

GitHub fires a deployment_status event on your repository as Vercel's integration creates and updates the deployment. Filter it for the moment the deployment finished successfully:

on:
  deployment_status:

jobs:
  e2e:
    if: github.event.deployment_status.state == 'success' &&
        github.event.deployment_status.environment == 'Preview'
    runs-on: ubuntu-latest
    steps:
      - name: Capture the preview URL
        run: echo "PREVIEW_URL=${{ github.event.deployment_status.target_url }}" >> $GITHUB_ENV

That target_url field is the commit-specific preview URL. Everything downstream, Playwright, Kane CLI, or any other tool, points at $PREVIEW_URL from here.

The Protection Bypass Every Guide Gets Half Right

Vercel's Protection Bypass for Automation documentation exists because Deployment Protection, Vercel Authentication, Password Protection, or Trusted IPs, is on by default for most teams and blocks exactly the traffic a test runner sends.

Vercel automatically sets a secret as the VERCEL_AUTOMATION_BYPASS_SECRET system environment variable once bypass is configured for the project. The documented, recommended method is sending it as a header:

• Header (recommended) - x-vercel-protection-bypass: <secret> on the request, the right choice for most automation.
• Query parameter - reserved for tools that can't set custom headers, like third-party webhook verification (Slack, Stripe, GitHub).

Here's the part most guides skip: a single header only clears the one request it's attached to. A real browser session, the kind an E2E test drives, makes many requests as it navigates. For that, Vercel's docs specify a second header, x-vercel-set-bypass-cookie: true, which sets the bypass as a cookie via a redirect so every subsequent page load in the session stays authorized. Inside an iframe, the value needs to be samesitenone instead.

Wiring It Into GitHub Actions

Vercel's own documentation ships a Playwright example using exactly these two headers together:

// playwright.config.ts
import { defineConfig } from '@playwright/test';

if (!process.env.VERCEL_AUTOMATION_BYPASS_SECRET) {
  throw new Error(
    'VERCEL_AUTOMATION_BYPASS_SECRET is required to run tests against protected deployments',
  );
}

export default defineConfig({
  use: {
    extraHTTPHeaders: {
      'x-vercel-protection-bypass': process.env.VERCEL_AUTOMATION_BYPASS_SECRET,
      // Use 'samesitenone' instead of 'true' when testing in an iframe.
      'x-vercel-set-bypass-cookie': 'true',
    },
  },
});

Combined with the deployment_status trigger from earlier, the full loop is: capture $PREVIEW_URL, run Playwright with these headers against it, and the run neither times out on a login screen nor accidentally tests the wrong commit.

Verifying With Kane CLI

Writing and maintaining a Playwright spec is the right call for a stable, well-defined flow. For a fast smoke check on every single preview, Kane CLI from TestMu AI skips the spec file: a plain-English objective, run headless with the same bypass header, against the same captured URL.

npm install -g @testmuai/kane-cli
kane-cli login --username "$LT_USERNAME" --access-key "$LT_ACCESS_KEY"

kane-cli run --agent --headless \
  --url "$PREVIEW_URL" \
  --header "x-vercel-protection-bypass: $VERCEL_AUTOMATION_BYPASS_SECRET" \
  "sign in with the test account, open the dashboard, assert the page
   shows the account name"

Agent mode's standard exit codes (0 passed, 1 failed, 2 environment error, 3 timeout) drop straight into the same GitHub Actions job as a required check, no separate reporting step. Our dedicated guide to connecting Kane CLI to GitHub Actions covers the full workflow setup, and the Kane CLI introduction documentation covers authentication and the full command reference.

Custom Environments for Staging and QA

Beyond the default Preview environment, Vercel's Pro plan allows one custom environment per project, and Enterprise allows twelve, per the current documentation. A custom environment like staging can track a specific branch automatically and hold a persistent domain, unlike the constantly-changing default Preview URLs.

A longer-running QA environment is the right target for a fuller regression pass; the per-commit preview URL from earlier is the right target for a fast smoke check on every single change.

Common Failures and Fixes

• Every test fails on a login page - Deployment Protection is on and the bypass header is missing or the secret is stale after a project setting change.
• The first request passes, later navigations fail - the header alone isn't enough for a multi-page session; add x-vercel-set-bypass-cookie.
• Tests run against the wrong version of the app - the job used a branch URL instead of the commit-specific target_url, and a later push changed what it served mid-run.
• The workflow never fires - the deployment_status filter checked the wrong environment name; Vercel reports it as exactly "Preview," case-sensitive.

Getting Started

Four steps from zero to a preview check that actually runs.

• Enable Protection Bypass for Automation on the Vercel project and confirm VERCEL_AUTOMATION_BYPASS_SECRET appears in deployment environment variables.
• Add the deployment_status trigger to a GitHub Actions workflow and capture target_url as an environment variable.
• Send both bypass headers, protection-bypass and set-bypass-cookie, from whichever tool runs the check.
• Start with one smoke objective on the flow that would hurt most if it broke, via Kane CLI or Playwright, before expanding coverage.

Once the loop runs reliably on every preview, our guide to agent-native CI covers the next step: having an agent triage what actually failed, not just report that it did. If the same check is gating an AI-generated pull request specifically, our breakdown of quality gates for AI pull requests covers where this fits in that broader review process.