The Kane CLI .evidence Format

The .evidence pack is not specific to Kane CLI. It is an open format that works with any test framework, with its own specification, validator, and Apache-2.0 licensed tooling, published at github.com/LambdaTest/evidence-cli. A pack has the same shape whatever produced it: a browser agent, a Playwright suite, a Jest run, or an API check. A CI dashboard, an auditor, or a person can read it without knowing which framework wrote it.

Who writes what

Knowing who writes each part explains what the format guarantees and what it does not.

Part of the pack	Written by
`run.yaml`, each `result.yaml`, the test definition, the per-step folders and screenshots, and the console, network and runner logs	kane-cli
The derived totals, each definition hash, the run-level failure index, and the seal	`evidence finalize`

The evidence-cli tooling captures nothing. It does not drive a browser or run tests. It validates a pack, computes the values that can be derived from its contents, and seals it. A screenshot is in your pack because Kane CLI put it there. This split is deliberate. Because the container is open, any tool can produce a pack and any tool can read one.

Profiles

Profiles are levels of a single contract. L1 adds requirements on top of L0 and never changes what L0 requires.

Profile	Requires
L0	The minimal core: a top-level `run.yaml`, and per test a definition file plus a `result.yaml`.
L1	All of L0, plus, once the run is finalized, the captured artifacts: declared logs, the `steps/` directories, a `coverage/` directory, and the run-level failure index. Video is optional. Before the run is finalized, L1 checks only the shape of what is already present.

The format treats the definition file as opaque: it references and hashes the file but never parses it. The file can be a test.md from Kane CLI, a login.spec.ts from Playwright, or anything else.

The contract version and the profile are independent. The version, 0.1, defines what the fields mean. Adding a profile only adds requirements and does not change the version. The version changes only when the meaning of an existing field changes in a breaking way.

What sealing does and does not give you

finalize adds up the totals, writes the content hash of each definition, sets the run to finalized, and seals the directory into the zip. Sealing replaces the live directory in a single atomic step, so a complete copy exists at every moment. The recorded definition hash protects the integrity of what was tested: validate fails a finalized pack if its definition does not match the hash recorded when the pack was sealed. Packs are not signed, so the hash says nothing about who produced the pack.

Using the format directly

To produce or read packs outside Kane CLI, install the tooling from npm:

npm install -g @testmuai/evidence-cli

evidence validate my-run.evidence --profile L0
evidence finalize my-run.evidence/

The standalone CLI provides validate, finalize, and merge. Exit codes differ by command:

validate: 0 valid, 1 invalid, 2 usage error.
merge: 0 merged, 1 policy abort, 2 usage error.

By default it reads its config from ~/.testmuai/evidence/config.json. Override the location with --config or the EVIDENCE_CONFIG environment variable. The active profile comes from the --profile flag first, then the config, then the built-in default of L0. The package is also a library, so you can call validate and finalize from your own code:

import { validate, finalize } from "@testmuai/evidence-cli";

const report = await validate("my-run.evidence", { profile: "L1" });
if (!report.valid) {
  for (const d of report.diagnostics) {
    console.error(`${d.severity} ${d.location}: ${d.message} [${d.code}]`);
  }
}

The standalone evidence CLI defaults to the L0 profile, while kane-cli evidence validate defaults to L1.

Next steps

Pack structure: the layout in full.
Validating packs: the checks the validator runs.

​Who writes what

​Profiles

​What sealing does and does not give you

​Using the format directly

​Next steps

Who writes what

Profiles

What sealing does and does not give you

Using the format directly

Next steps