GPT-6 and Intelligent UI: How to Test AI-Generated Apps

Step 1: Write the Definition of Done Into AGENTS.md

OpenAI's model guide tells teams to be prescriptive about persistence: define what done includes, from implementing the change to running it, inspecting the result, and fixing failures. It also suggests authorizing safe routine workflows in AGENTS.md, such as running local tests with disposable data and no production access.

The Using GPT-6 guide gives a reason to get the wording right: GPT-6 Astra can be more sensitive to instructions in skills and files such as AGENTS.md, and OpenAI recommends auditing those files for instructions that could influence its behavior.

A block like this one makes the browser check part of done:

## Definition of done for UI changes
- Run unit tests with disposable data and no production access.
- Start the dev server, then run a Kane CLI check for each user flow the change touches:
  kane-cli run "Go to http://localhost:3000/<page> and <expected outcome>" --agent --headless
- The task is done only when the run_end event reports "status": "passed".
- On a failure, read the run summary and the evidence pack, fix the code, and rerun.
- Ask before editing a check's objective to make it pass.

The last rule matters because an agent that can edit its own checks can also make them pass. The same guide's suggested prompt tells the model to skip tests that mirror the implementation for reversible, low-impact changes, so the browser check is the one that has to describe what the user sees.

Step 2: Give Codex a Browser Check

Kane CLI is a browser agent from TestMu AI that validates rendered UI in a real Chrome browser from natural-language objectives, so Codex checks what the user sees rather than what the code declares. Install the CLI and the agent skill once:

npm install -g @testmuai/kane-cli
npx @testmuai/kane-cli-skill

According to the Kane CLI with AI coding agents docs, the skill installs for Claude Code, Codex CLI, and Gemini CLI in one command. Codex then runs objectives in agent mode, where Kane CLI writes one JSON event per line. A check on a bill splitter Codex just built looks like this:

kane-cli run "Go to http://localhost:3000/split, fill the bill amount with 120, set the number of people to 4, and assert the per-person share shows 30.00" --agent --headless

The stream ends with a run_end event, the one event a script or agent should read. Its documented shape:

{
  "type": "run_end",
  "status": "passed",
  "summary": "Filled the bill splitter and confirmed the per-person share",
  "reason": "Objective completed",
  "final_state": {},
  "session_dir": "~/.testmuai/kaneai/sessions/<session-id>"
}

Each step names the user-facing element, such as "the per-person share", so autoheal re-resolves it when a class name or layout changes. Matches carry a confidence score, and low-confidence matches are rejected, so a genuinely ambiguous change is surfaced for review instead of being healed past. If Codex is new to your team, the Codex CLI guide covers its sandbox and approval modes.

Step 3: Gate the Merge on the Evidence

In CI, the same command becomes a merge gate, because Kane CLI's exit code carries the verdict:

• Exit code 0 - the objective passed.
• Exit code 1 - the objective ran and the app did not meet it. Block the merge.
• Exit code 2 - an auth, setup, or infrastructure error, such as Chrome or the network. Fix the environment and rerun before anyone files a product bug.
• Exit code 3 - a timeout or cancellation.

Each run also saves an evidence pack with step traces, screenshots, video, network and console logs, and the full NDJSON stream. Attach it to the pull request so the reviewer sees what happened in the browser next to the agent's summary.

Baseline the Component Library

A generated layout has no stable baseline, but the components it is built from do. Intelligent UI draws on a library of native components, and a generative UI of your own works best with the same constraint: a fixed set of cards, charts, forms, and calculators that the model fills and arranges.

Capture each component in each state, such as empty, loading, filled, and error, and compare it with an approved baseline on every change. SmartUI, TestMu AI's visual AI testing platform, captures every Storybook story with the smartui storybook command, and its Smart Ignore suppresses expected changes such as loading states and rotating content. The visual regression testing guide covers baselines and approvals in depth.

Assert on Outcomes and Accessible Names

On a generated screen, assert what the user can do and see: the total a calculator returns, the label on the button that submits a form, the error a form shows for a missing field. Locate elements by role and accessible name, since the DOM path changes whenever the model picks a different arrangement.

Plain-English objectives in Kane CLI already work at this level, for example "assert the Split button is enabled" or "set the number of people to 0 and assert an error message appears". Those assertions still hold when the model places the button in a layout it has never produced before.

Wait for the Stream to Settle

OpenAI's compiler renders the interface progressively, before the response is complete, and GPT-6 can begin answering while it keeps thinking. A check that fires at first paint can pass on a half-built card or fail on a chart that was still loading.

Wait for a completion signal your app controls, such as a done flag on the response or the final component's ready state, and avoid fixed sleeps. Kane CLI's vision-based dynamic waiting watches the rendered viewport and acts when the screen is ready, which covers streamed UI where the DOM settles before the visuals do.

Sample Each Prompt More Than Once

GPT-6.1 Sol and GPT-6 Astra do not take a temperature setting at the reasoning efforts they support, according to the Using GPT-6 guide, so output variation cannot be pinned down from the request side. One passing run shows that one arrangement worked.

Run each test prompt several times and require every run to hold the same invariants: the right component type, correct values, no console errors, and a working path to the user's goal. Record which components the model chose in each run, so a change in layout choices after a model update shows up as data.

Check the Values Inside the Answer

A savings calculator or a bill splitter in a generated answer shows numbers the user will act on. Compute the expected value in code from the same inputs and compare it with what the interface shows, since a well-rendered card with a wrong total is still a failed test.

For the text around the interface, Agent Testing from TestMu AI scores chat answers on 9 quality dimensions, including hallucination and completeness. Its Data Validation checks values an agent states, such as a price or an order status, against your own system of record through read-only API lookups.