Can Coding Agents Test Their Own Code

Can a Coding Agent Test Its Own Code

Partly, and the split is predictable. It catches faults that contradict the code in front of it, and misses faults that require knowing what the code was supposed to do.

That distinction is worth more than a general opinion about whether AI can be trusted. It tells you exactly which reviews you can drop and which you cannot.

What Happens When One Agent Writes Both

The code and the test inherit one reading of the requirement. If that reading is wrong, the test does not disagree with the code. It confirms it, and the suite goes green.

Statisticians would call this correlated failure. Two checks that share an input are not two checks.

• The proofreader problem - an author reads past their own errors because they know what the sentence meant to say.
• The discount field - nobody stated that a discount cannot exceed the order total, so nothing written was going to check it.
• Volume does not help - ten tests from one misreading are still one misreading.

This is why coverage percentages mislead on agent-written suites. High coverage of the wrong intent is still the wrong intent.

What Independent Verification Actually Requires

Independence is not about who runs the check. It is about whether the thing confirming the work also produced it, and whether the result survives outside the session.

Three properties separate evidence from a self-report.

• Executed, not reasoned - the check ran against the running application rather than against the diff.
• Durable - the result is a file in your repository, not terminal output that scrolls away.
• Attributable - you can see which steps ran, in what order, and what each one asserted.

A deterministic verifier gives you all three without adding a second reasoning layer to supervise. Kane CLI from TestMu AI takes a natural-language objective and drives a real Chrome browser to a pass or fail.

• Separate from the author - the pass is decided by observable evidence, not by the system that wrote the code.
• One portable pack - results, per-step screenshots, and console output travel in a single evidence file.
• Fails the build - a failing verification stops the pipeline before agent-written code reaches main.
• Pushes through surprises - redirects and error pages are treated as data rather than reasons to stop.

Installed as a skill, the coding agent calls it after building something and reads the verdict back.

In practice the call is one line, and the objective states the rule nobody wrote down.

# the coding agent invokes this itself, then reads the result
kane-cli run --agent --headless \
  "apply a discount larger than the order total,
   assert the form rejects it and the total never goes below zero"

# exit 0 = the rule holds. exit non-zero = the agent shipped the bug it could not see.

Setup and the command reference live in the Kane CLI introduction documentation. For background and scheduled work, we covered the pattern in async agents and Kane CLI.

Note what the objective above contains. It states a rule the agent was never told, which is the part a human still has to supply.

Which Failures Self-Testing Actually Catches

Sorting failures by whether the agent already had the information needed to spot them gives a usable rule.

Read the last two rows as your remaining job. Everything above them is increasingly safe to delegate.

Why a Real Browser Changes the Answer

One row in that table moves depending on your setup. Broken rendering is invisible to an agent reasoning about source code and obvious to one watching a page.

A unit test proves a function returned a value. It says nothing about whether the element rendered, stayed visible, or survived the stylesheet that shipped alongside it.

• Overlap - a new banner covers the submit button. Every unit test still passes.
• Race on load - the handler binds after first paint, so the first click does nothing.
• Silent console error - a third-party script throws and the page half-renders.
• State that never clears - a modal closes visually and keeps intercepting clicks.

None of those appear in a diff. All of them appear in the first ten seconds of driving the page.

This is the cheapest upgrade available, because it converts a class of invisible failures into visible ones without changing how the agent works.

It still does not touch the bottom two rows. A browser shows you what happened, not what should have happened.

Where a Second Agent Helps and Where It Does Not

A common answer to this problem is to add a dedicated testing agent. It helps in some places and quietly fails in others.

• Helps on fresh eyes - a second agent reading only the requirement can spot what the first one skipped.
• Helps on breadth - it explores edge cases the implementing agent had no reason to consider.
• Fails on shared context - if it reads the same ticket and the same code, it inherits the same blind spot.
• Fails on accountability - two non-deterministic systems agreeing is not proof, it is a coincidence with better odds.

The useful version gives the second agent something the first did not have. That usually means the specification, the ticket, or a real user session.

Our comparison of Claude Code vs Antigravity covers how two harnesses differ on the evidence they leave behind, which matters more here than model quality.

How to Set a Human Gate That Works

Reading every agent-written line does not scale, and it spends attention in the wrong place. Gate on blast radius instead.

• Let the agent merge low-risk, reversible changes when the evidence file passes.
• Require a human decision on payments, auth, data deletion, and migrations.
• Review the evidence rather than the diff, and ask what the assertions proved.
• Treat a healed test as a review trigger, since healing can hide a real regression.

The fourth point catches teams out. A suite that repairs itself will happily repair away the signal you needed.

Gating by blast radius rather than by author also survives scale. The rule does not change when the agent goes from writing ten files a week to a hundred.

Teams that skip this end up in one of two failure modes. Either everything needs approval and the agent stops being useful, or nothing does and the first bad merge is expensive.

What to Change First

So the answer to the title is a qualified yes, and the qualification is the whole point.

An agent can test its own code for everything except whether the code was the right thing to build. Three changes move the line.

• Put a real browser behind every UI change, so rendering failures stop being invisible.
• Store verification artifacts in the repository, so evidence outlives the session.
• Write the unstated rules down, because that is the only gap tooling cannot close.

The third one is unglamorous and does the most work. Every requirement you leave implicit is a test nobody will write.

For scoring behaviour that varies between runs, our guide to AI agent testing covers evaluation beyond a single pass or fail.