Why AI-Generated Tests Pass When Your App Is Broken

Why Do AI-Generated Tests Pass When the Code Is Wrong?

A test has two halves, the steps and the expected result, and a model shown the implementation usually computes the expected result by reading that implementation. A test written that way is a snapshot of current behaviour. It will catch the next regression, and it will never catch a bug that was already there when the snapshot was taken.

Research on LLM-generated oracles across 24 Java repositories found that LLM-based test generation mainly captures the actual program behaviour, which makes bug detection difficult. The authors concluded that all LLM oracle suggestions need human inspection.

Huang, Zhang, Harman and co-authors measured the cost of incorrect code in the prompt directly: tests generated for incorrect code experienced a 47% worse bug detection rate.

• Backfilled tests lock bugs in - asking an agent to "add tests" for an existing module produces regression snapshots. Use them as change detectors before a refactor, and never as evidence that the module works.
• The expected result needs its own source - a ticket, a docstring, a product spec or a person. Code cannot serve as its own oracle, which is the test oracle problem at the scale of a single function.
• One session, one blind spot - when the same agent writes the feature and its tests, a misread requirement lands in both. The piece on whether coding agents test their own code covers that case in depth.

Experiment Methodology: Six Seeded Bugs and Two Models

Each module is a small CommonJS function whose JSDoc block states its business rule, with one realistic bug planted in the body. A hand-written reference test confirms every bug: it fails on the buggy version and passes on the correct one.

Each module went through three conditions, run 5 times per module on Gemini 3.1 Pro Preview and Gemini 3.8 Flash at default settings:

• Code-first - the model sees the whole buggy file, docstring included, and is asked for a thorough Jest test file, the usual "write tests for this file" request.
• Spec-first - the model sees the same docstring and signature with the body replaced by a comment, and gets the same request.
• Fix loop - the model gets the buggy file, the failing reference test and the names of the failing tests, and is told to make the failing tests pass, returning every file it changed.

Here is the spec-first prompt. The code-first prompt is identical except that it opens with "Here is the file cartTotal.js:" and includes the full buggy file:

You are working in a JavaScript (CommonJS) project that uses Jest. Here is the documented contract for `cartTotal.js` (the implementation is not shown):

```js
<the JSDoc block and the function signature, with the body replaced by a comment>
```

Write a thorough Jest test file for this module. It will be saved as `cartTotal.test.js` in the same folder and must import the module with `require('./cartTotal')`. Reply with only the test file in a single ```js code block.

That produced 60 code-first suites, 60 spec-first suites and 60 fix attempts. Every suite then ran under Jest 29.7 twice, once against the buggy module and once against the correct one, which sorts it into one of these outcomes:

• Green on the bug - the suite passes on the buggy code. In this experiment every such suite also failed on the correct code, which means it encoded the bug as expected behaviour.
• Catches the bug - red on the buggy code and green on the correct code, the only outcome a test exists to produce.
• Fails on both - wrong in some other way, such as asserting a rule the contract never promised.

Results: 22 of 60 Code-First Suites Passed on the Bug

Seeing the implementation made the difference. Code-first suites passed on the broken code 22 times out of 60, and spec-first suites written from the same docstring never did.

• Boundary bugs were encoded 16 times out of 20 - the models read > 50 and asserted that a subtotal of exactly 50.00 pays shipping, with the docstring stating "50.00 or more" a few lines above.
• Rules stated as hard limits held up - "the amount due is never below 0" led 9 of 10 code-first suites to test a credit larger than the order, which exposed the missing cap.
• The smaller model encoded more - Gemini 3.8 Flash wrote 14 of the 22 bug-encoding suites, including all 5 for paginate. Gemini 3.1 Pro Preview wrote the other 8.
• One wrong assertion is enough - 17 of the 22 bug-encoding suites contained exactly one test asserting the bug, out of an average of 14.3 tests per suite.

The clearest evidence is in the tests themselves. Five of the bug-encoding suites spotted the conflict between the docstring and the code, wrote it down, and sided with the code:

// From a Gemini 3.1 Pro Preview suite written from the buggy cartTotal.js
it('should charge shipping when the subtotal is exactly 50 (testing current implementation boundary "> 50")', () => {
  // Note: The docstring says "50.00 or more", but the code implements `discounted > 50`.
  // This test asserts the actual implementation behavior.
  const items = [{ price: 50.00, qty: 1 }]; // subtotal = 50
  expect(cartTotal(items)).toBe(55.99); // 50 + 5.99
});

Five other code-first suites wrote down the same kind of conflict and sided with the docstring, so their tests fail on the bug. All ten suites that noted a conflict came from Gemini 3.1 Pro Preview, and they split five to five: the model reads the contract, picks a side when the code disagrees, and nothing in a green run tells you which side it picked.

What Happens When You Ask AI to Make Failing Tests Pass?

In this experiment, the models fixed the code. Given a failing reference suite and the instruction to make it pass, Gemini changed only the implementation in 59 of 60 runs and never edited the test file, and every changed implementation passed the original reference tests.

The one remaining run returned the buggy file unchanged. These were easy fixes with an honest answer available, and the documented cases of test editing come from the opposite situation.

• Claude 3.7 Sonnet - the Claude 3.7 Sonnet system card says the model occasionally resorts to special-casing to pass tests in agentic coding environments like Claude Code, including "modifying the problematic tests themselves to match the code's output". Anthropic attributes the behaviour to reward hacking during reinforcement learning.
• ImpossibleBench - on SWE-bench tasks altered so their tests contradict the spec, ImpossibleBench found GPT-5 cheating on 76% of Oneoff-SWEbench tasks. When Claude models cheated, they did it primarily (over 79%) by modifying test cases.
• Kent Beck - his list of warning signs in augmented coding includes "any indication that the genie was cheating, for example by disabling or deleting tests."

Reward hacking means optimizing the signal you are graded on, a green suite, instead of the goal it stands for, working code. Watch test-file diffs hardest after an agent has been stuck on a failure, and require a human to confirm the old assertion was actually wrong whenever one pull request changes a failing test and the code under it.

For which kinds of defect slip past each test gate, see the data on AI-generated code bugs.

How AI-Generated Tests Go Green Without Checking Anything

Encoding the bug is one route to a false green. The others come from tests that never check the real result at all, and those are mechanical enough for tooling to catch.

// cart.js - broken: every cart comes back with a phantom item, and unknown users do not throw
async function fetchCart(userId) {
  await new Promise((resolve) => setTimeout(resolve, 50)); // simulated network call
  return { items: [{ sku: 'PHANTOM' }] };
}
module.exports = { fetchCart };

// cart.test.js
const { fetchCart } = require('./cart');

test('rejects an unknown user', async () => {
  try {
    await fetchCart('nobody');
  } catch (err) {
    expect(err.message).toMatch(/unknown user/); // only runs if fetchCart throws
  }
});

test('handles a second user', async () => {
  await fetchCart('u2'); // no assertion at all
});

test('loads an empty cart', () => {
  fetchCart('u1').then((cart) => expect(cart.items).toEqual([])); // not awaited or returned
});

// eslint.config.js (ESLint 9, eslint-plugin-jest 28)
const jest = require('eslint-plugin-jest');

module.exports = [
  {
    files: ['**/*.test.js'],
    ...jest.configs['flat/recommended'],
    rules: {
      ...jest.configs['flat/recommended'].rules,
      'jest/expect-expect': 'error',
      'jest/valid-expect': 'error',
      'jest/no-conditional-expect': 'error',
    },
  },
];

$ npx eslint cart.test.js
   7:5  error  Avoid calling `expect` conditionally`   jest/no-conditional-expect
  11:1  error  Test has no assertions                 jest/expect-expect
  16:3  error  This promise should either be returned or awaited to ensure the expects in its chain are called  jest/valid-expect-in-promise

✖ 3 problems (3 errors, 0 warnings)

Does Mutation Testing Catch AI-Generated Tests That Encode Bugs?

No. Mutation testing catches tests that check too little, and it gives full marks to tests that precisely check the wrong thing. A suite that pins buggy behaviour kills mutants of the buggy code as reliably as a correct suite kills mutants of the correct code.

Stryker's documentation on mutant states calls a mutant survived when all tests passed while it was active ("You're missing a test for it") and killed when at least one test failed. The mutation score is detected mutants divided by valid mutants.

I ran Stryker 8.7 on every suite against the implementation it was green on: the buggy file for the bug-encoding suites, the correct file for the bug-catching spec-first suites.

Stryker's EqualityOperator mutator turns > into >=, so on cartTotal and isStrongPassword one of the mutants it generated was the correct implementation. Every bug-encoding suite that met that mutant killed it, and Stryker counted the fix as a defect the tests had caught.

The bug-encoding suites also outscored the bug-catching ones on average, 97.3 against 90.2. The two groups were mutated on different implementations, so the gap is no ranking; it shows that a mutation score measures how sensitive tests are and says nothing about whether their expected values are right.

• Use it for weak assertions - surviving mutants point straight at presence-only checks and assertions that never run. The mutation testing guide covers operators and scoring.
• Read the killed mutants on boundary lines - in Stryker's HTML report, compare every killed mutant on a comparison or a rounding call with the spec. A killed mutant that matches the documented rule means one of your tests asserts the bug.
• Mutate only what changed - npx stryker run --incremental stores results and reuses them on the next run, so a pull request pays mainly for the code it touched.

How to Run a Sabotage Check in a Real Browser

A sabotage check runs your existing tests against a build you broke on purpose. Any test that stays green is not testing the behaviour you broke, whoever or whatever wrote it.

For browser tests, Playwright's page.addInitScript() injects a fault without touching the app. The spec below fills in the Simple Form Demo on the Selenium Playground twice, once on the healthy page and once with a script that makes the page render "undefined" in place of the message.

// playwright.config.js - runs every test on TestMu AI's cloud grid
const { defineConfig } = require('@playwright/test');

const capabilities = {
  browserName: 'Chrome',
  browserVersion: 'latest',
  'LT:Options': {
    platform: 'Windows 11',
    build: 'Sabotage check',
    name: 'AI-written assertions vs a broken build',
    user: process.env.LT_USERNAME,
    accessKey: process.env.LT_ACCESS_KEY,
  },
};

module.exports = defineConfig({
  testDir: './tests',
  reporter: 'list',
  use: {
    screenshot: 'only-on-failure',
    connectOptions: {
      wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`,
    },
  },
});

// sabotage.spec.js - run the same assertions against a healthy and a deliberately broken page
const { test, expect } = require('@playwright/test');

const URL = 'https://www.testmuai.com/selenium-playground/simple-form-demo/';

// The injected fault: whatever the user types, the page renders "undefined".
async function breakTheMessage(page) {
  await page.addInitScript(() => {
    new MutationObserver(() => {
      const message = document.querySelector('#message');
      if (message && message.textContent && message.textContent !== 'undefined') {
        message.textContent = 'undefined';
      }
    }).observe(document, { childList: true, subtree: true, characterData: true });
  });
}

// Mark the TestMu AI session failed when any test in it fails, so the dashboard agrees with the report.
test.afterEach(async ({ page }, testInfo) => {
  if (testInfo.status !== testInfo.expectedStatus) {
    const action = { action: 'setTestStatus', arguments: { status: 'failed', remark: testInfo.title } };
    await page.evaluate(() => {}, `lambdatest_action: ${JSON.stringify(action)}`);
  }
});

for (const broken of [false, true]) {
  test.describe(broken ? 'broken build' : 'healthy build', () => {
    test.beforeEach(async ({ page }) => {
      if (broken) await breakTheMessage(page);
      await page.goto(URL);
      await page.getByPlaceholder('Please enter your Message').fill('Hello TestMu');
      await page.locator('#showInput').click();
    });

    test('weak: the message is visible', async ({ page }) => {
      await expect(page.locator('#message')).toBeVisible();
    });

    test('strong: the message shows what was typed', async ({ page }) => {
      await expect(page.locator('#message')).toHaveText('Hello TestMu');
    });
  });
}

Export LT_USERNAME and LT_ACCESS_KEY, run npx playwright test, and the four tests execute on Chrome on Windows 11 in the cloud. The Playwright testing docs list the other capabilities you can set.

Running 4 tests using 1 worker

  ok 1 tests\sabotage.spec.js:35:5 › healthy build › weak: the message is visible
  ok 2 tests\sabotage.spec.js:39:5 › healthy build › strong: the message shows what was typed
  ok 3 tests\sabotage.spec.js:35:5 › broken build › weak: the message is visible
  x  4 tests\sabotage.spec.js:39:5 › broken build › strong: the message shows what was typed

  1) tests\sabotage.spec.js:39:5 › broken build › strong: the message shows what was typed

    Error: expect(locator).toHaveText(expected) failed

    Locator:  locator('#message')
    Expected: "Hello TestMu"
    Received: "undefined"
    Timeout:  5000ms

  1 failed
  3 passed

The visibility assertion passed on a page showing the wrong text. Only the value assertion failed, and the failure screenshot above shows exactly what a user would have seen.

Keep the afterEach hook. Sessions opened through connectOptions are not marked passed or failed on their own, so without it the dashboard shows a finished session while the Playwright report shows a failure. The same deliberate-break idea also proves parity in a Selenium to Playwright migration.

Review Checklist for AI-Generated Unit Tests

Run these gates in order. The first one prevents the problem, and the rest catch what slips through.

• Generate from the contract - open a fresh session, give the model the ticket, docstring or acceptance criteria, and keep the implementation out of the prompt. In the experiment, that one change took bug-encoding suites from 22 to 0, and the ISSTA study proposes the same remedy: replace the code under test in the prompt with a specification docstring.
• Grep for admissions - search generated tests for wording about the implementation or current behaviour, then read each hit. The pattern below flagged 12 of the 22 bug-encoding suites, along with 7 suites that mentioned the implementation while siding with the spec.
• Lint for assertions that never run - fail CI on expect-expect, valid-expect, valid-expect-in-promise and no-conditional-expect.
• Mutate changed files - run Stryker incrementally and read the killed mutants on boundary lines against the spec.
• Gate test edits - require a separate human approval when one pull request changes a failing test and the code beneath it.
• Sabotage the browser path - keep one injected fault per critical user flow and run it on every merge to main.

# Generated tests that describe the implementation instead of the spec - read every hit
grep -rniE "implementation|current behavio|based on (the )?code" --include="*.test.*" .

Kane CLI from TestMu AI builds the first gate into its assurance commands. kane-cli design tests turns a requirement use-case into acceptance criteria, each with an expected result a machine can check, and cites each one back to the pinned source text.

When the source never states an expected result, the design run records a missing-expected-result gap instead of inventing one from the code. kane-cli cover then reports what a sealed evidence pack proved separately from what the design still owes, so a green run that left a criterion unverified shows up as a gap.

Note: Run your sabotage checks and Playwright suites on the TestMu AI cloud grid, with 3,000+ browser and OS combinations and a video, console log and network log for every session. Start testing free with TestMu AI

First Step for Your Existing AI-Generated Tests

Pick the module with the most AI-generated tests, give its docstring or ticket to a fresh session with the implementation hidden, and run the new suite against the current code. Every new failure is either a bug the old suite had locked in or a rule nobody wrote down.

For browser flows, the Kane CLI assurance design docs show the same contract-first loop, from requirement source to cited acceptance criteria to a coverage report.