How to Set Up a Vibe Coding QA Workflow

What the Data Says About Agent-Written Code

Developers are using these tools heavily and trusting them less each year. The 2025 survey shows adoption holding steady while confidence in output accuracy falls.

Four figures from that survey frame the problem a QA process has to solve.

Read the second row against the first. The same tool that speeds up writing is slowing down debugging for nearly half of developers.

That is the trade a process has to fix. Faster output is only a gain if verification keeps pace with it.

Why Vibe Coding Breaks Traditional QA

Conventional QA is built on sampling. You cannot test everything, so you test what looks risky and accept the rest.

Sampling assumes a human wrote the change and can explain the reasoning. Three of its assumptions stop holding when an agent authored the diff.

• Volume outruns reading - nobody reviews a thousand generated lines a day with real attention.
• Intent is not in the diff - the reviewer reconstructs what was wanted instead of checking it.
• Confidence is uniform - the agent presents a guess and a certainty in exactly the same tone.

So the model flips from sampling to verifying. Every change gets an automatic check, and human attention is spent only where a mistake is expensive.

For the underlying shift in practice, our explainer on what is vibe coding covers how the authoring side changed.

Which Stages Actually Block a Merge

Published versions of this process run to nine stages. Three of them can stop a bad change reaching production, and the other six make the process better without holding anything up.

Sorting them that way matters, because a team adopting nine stages at once usually abandons all nine.

The cost column explains the sequencing. The three blocking gates are cheap, and the advisory stages are the ones that need budget and calendar time.

How to Add the Three Blocking Gates

Gate one is a sentence, written before the agent starts, describing what a user should be able to do afterwards.

It sounds trivial and it is the load-bearing part. Everything downstream compares the result against that sentence.

• Gate two runs the app - the check drives the running application rather than reasoning about the diff.
• Gate two is issued elsewhere - something other than the agent that wrote the code has to call it.

Kane CLI from TestMu AI fits this slot because the objective and the intent sentence are the same artifact.

• Objectives read like intent - the sentence you wrote in gate one becomes the command you run.
• Verdict, not narration - a run ends in a pass or a fail rather than a written summary.
• Pipeline-ready - the headless flag makes the same check a merge gate.
• Portable output - runs export to Playwright when a team wants the suite in its own code.

# gate one: the intent, written before the agent starts
#   "a signed-out visitor can apply a promo code and see the discount"

# gate two: the same sentence, executed against the running app
kane-cli run --agent --headless \
  "as a signed-out visitor, apply promo code SAVE10 at checkout,
   then assert the order total drops and no login prompt appears"

Gate three is a branch rule rather than a tool. Anything touching auth, payments, deletion, or a migration waits for a named human.

Setup lives in the Kane CLI introduction documentation. Why the check has to come from outside the agent is argued in can coding agents test their own code.

What the Advisory Stages Add

The remaining six are worth having. They are just the wrong place to begin, because none of them stops a bad merge on its own.

• Contracts - typed boundaries reduce how much the agent can misinterpret in the first place.
• Agent-generated tests - they add breadth cheaply, as long as nothing treats them as proof.
• Staging exploration - finds the paths your written intent never mentioned.
• Production monitoring - tells you which flows real users actually depend on.
• Regression learning - every production bug becomes a permanent check.
• Review conventions - agreeing what a reviewer looks at stops review becoming theatre.

Regression learning compounds the fastest. Each production escape you convert into a check is one the agent cannot repeat.

For the framework-level view of this, our guide to vibe testing covers the principles underneath these stages.

Who Owns Each Stage

A stage without a named owner does not happen. The useful question is not which team is responsible, it is who produces the artifact.

Notice that no row says QA team. Treating this as a downstream inspection step recreates the bottleneck the agents were meant to remove.

The artifact column is the accountability mechanism. If a stage produces nothing you can point at, nobody can tell whether it ran.

Where Most Teams Actually Sit

Maturity here is not how many stages you have drawn on a diagram. It is how many of the three blocking gates a change cannot get past.

• Level zero - agent output merges after a human skim. Common, and it does not survive volume.
• Level one - a blast-radius rule exists, so the worst changes wait for a person.
• Level two - an executed check gates the merge, and green means something ran.
• Level three - intent is written first, so the check has something to be checked against.

Most teams sit at level one and believe they are at level two, because a passing test suite feels like verification whether or not anything ran the application.

The survey data suggests why the gap persists. Only 3.1% of 33,244 respondents said they highly trust AI output accuracy, and 45.8% actively distrust it.

Teams distrust the output and merge it anyway. A gate is what turns that unease into something the pipeline enforces.

The Pitfalls That Undo the Process

Four failure patterns show up repeatedly, and none of them looks like a failure while it is happening.

• Adopting all nine stages at once - the process collapses under its own ceremony within a month.
• Counting agent-written tests as verification - they are breadth, not proof, and they share the agent's misreading.
• Letting the suite heal silently - a test that adapts to a UI change can adapt past a real regression.
• Gating on coverage percentage - high coverage of the wrong intent is still the wrong intent.

The second one is the expensive mistake. It converts a genuine gap into a green dashboard, which is worse than having no gate at all.

Which Stage Teams Skip Most

Gate one, every time. Writing the intent down produces no artifact anyone demos, takes discipline rather than budget, and is the reason the other two gates work at all.

Skipping it looks harmless because the pipeline still runs. A verification with no stated intent behind it can only confirm that the code does what the code does.

A written intent takes about twenty seconds and has to survive three tests.

• User-facing - it says what a person can do, not which function changes.
• Falsifiable - someone could run it and disagree with the result.
• Written before - anything recorded afterwards describes the build rather than the requirement.

The third test is the one that gets skipped quietly. An intent written after the agent finishes is a summary, and a summary always agrees with the code.

Note: Intent is free. Execution is the part that needs a runner. Start verifying on TestMu AI free and put an executed check in your pipeline.

The second most skipped is the blast-radius rule, usually because it feels like distrust. It is not about trust, it is about who answers for the outage.

How Long This Takes to Stand Up

Any single number here is a guess, because the answer depends almost entirely on one thing: whether your application can already be driven from a command line.

• Already headless-testable - the three gates land in a few days on one repository.
• Testable but slow - budget a sprint, mostly for making the suite fast enough to gate on.
• Manual QA only - start with one critical flow rather than the whole application.

Scope it to a single repository first. A process proven on one service transfers, and a process designed for everything usually ships nowhere.

Where to Start Tomorrow

Pick the flow that would embarrass you most if it broke, then run these four steps against it.

• Write one sentence describing what a user should be able to do.
• Turn that sentence into a check something other than the agent can run.
• Put the check in CI so a failure blocks the merge.
• Add a branch rule requiring a human on destructive changes.

That is the whole process at minimum viable size. The other six stages are improvements you make once these three hold.

The survey number at the top is the reason to bother. Almost right is not a rare failure, it is the majority experience, and it is invisible to a check the agent wrote for itself.