Duplicate Test Case Detection With AI: Cut Repository Noise Without Losing Coverage

Why Test Suites Grow Faster Than Coverage

Writing a test case used to take real effort. That effort was a filter. You wrote a case when you needed one.

AI removed the effort, and nothing replaced the filter. Duplicates now land in review, and review was built to judge quality, not to spot a case the suite already has.

The 2025 DORA report sees the same pattern across software teams. In a survey of nearly 5,000 technology professionals, 90% of respondents report using AI at work. It also found AI adoption "does continue to have a negative relationship with software delivery stability". Its explanation is simple: AI "doesn't fix a team; it amplifies what's already there".

In a test repository, that amplification is coverage inflation. The case count grows faster than the behaviours, risks and requirements those cases actually test.

Case count is inventory. Real test coverage is the number of distinct behaviours you check. The two can drift apart for a year before anyone notices.

Four Kinds of Duplicate, and Two You Must Keep

A scan surfaces four kinds of match. They look alike in the results. They need different actions.

The bottom two rows are why nothing should be deleted automatically. A USD and an EUR payment case look redundant to any similarity measure. They protect different money flows.

So the rule for everything that follows: similarity decides what a reviewer looks at next. It does not decide what the repository forgets.

Detail makes that review fast. A test case with preconditions, steps and expected results is easy to judge. A bare title is a guess.

How Duplicate Detection Works in Test Manager

The obvious approach is text matching. It catches exact copies and one-word edits. Then it fails in two opposite ways.

It misses paraphrases. "Verify user can log in with valid credentials" and "Check successful login for registered user" test the same thing and share almost no words.

It over-matches variants. Swap USD for EUR and the text barely changes. The behaviour does.

Semantic search fixes the first problem by comparing meaning instead of words. Research backs it: Viggiato et al. combined text embeddings, similarity and clustering and reached an F-score of 83.47% in an industrial evaluation. That number belongs to their dataset, not yours. The approach holds.

The second problem needs a person. So the scan ranks. It never deletes.

What the scan does

• Reads each case - title, steps and description become a vector embedding.
• Finds neighbours - a nearest-neighbour search runs across the project or folder you pick.
• Scores matches - every candidate gets a % Match from 0 to 100, in four bands.
• Leaves the call to you - compare, ignore, or promote a better base case and recalculate the group.
• Costs the same at any size - 5 credits whether it covers 100 cases or 5,000, charged only when it finds duplicates.
• Runs one at a time - one scan per project. Scan large projects folder by folder.

The call gets made in the comparison view. It opens with a plain-language summary of how the two cases differ, then shows the diff, steps, field values and linked items. Check out this detailed documentation to deduplicate test cases and tune every scan setting.

Turning a Match Score Into a Decision

A match score is an ordering. It tells you where to look first. TestMu AI groups scores into four bands and recommends starting at 80 and above. The threshold changes how much lands in your queue. It does not make the scan more accurate.

Do not read 90% as "90% likely to be redundant". Nothing calibrates the score that way. Measure it instead: for each band, divide the candidates a reviewer confirmed by the candidates reviewed. That is your precision by band, and it tells you where your own cutoff belongs.

Delete or archive?

Delete when you are sure. Archive when you are not. The difference matters:

• Deletion is final - a case deleted through the scan cannot be restored, though its past test runs remain.
• Archiving loses nothing - the case keeps its ID, version history, fields, tags, run results and linked issues, and unarchiving returns it to its folder unchanged.
• Archiving still clears the noise - archived cases leave lists, search, runs, reports, AI generation context and future scans.
• Both are on the record - every archive and unarchive shows in test case versioning with the user and time.

Before you delete, read the footer. It splits everything you marked by band. Forty cases from above 80% is routine cleanup. Forty mostly from below 50% needs a second reviewer. Check out this detailed documentation to archive test cases, up to 500 at a time.

Prevention and Remediation Are Two Different Controls

A scan cleans up what exists. It has no say in what gets written next week. Teams that only scan end up scanning again two quarters later.

Two controls, two jobs

Smart Context, the memory layer, prevents duplicates from being created during AI generation. The duplicate scan finds duplicates that already exist in your repository.

One stops the inflow. The other drains the backlog. Scan without Smart Context and the backlog refills. Turn on Smart Context without scanning and the old duplicates stay. Both live in one test management platform, so the cases Smart Context reads are the same cases the scan cleans.

What Never to Merge, by Industry

The scan works the same everywhere. What changes is the pair that looks redundant and is not.

Regulated teams have the strictest rule, and it comes from the regulation. Where 21 CFR Part 11 applies, closed systems must keep records accurate and retrievable for the full retention period, plus time-stamped audit trails for anything that creates, changes or deletes a record. Changes "shall not obscure previously recorded information".

A permanent delete does not fit that. In regulated work, archive or export first, record the reviewer's approval, and delete only where your records policy allows it.

Cleaning the repository is not trimming the suite

Deduplication and test suite minimization get mixed up. They answer different questions.

Keep them separate. Deduplication decides which written cases survive. Minimization decides which executable tests run. Regression testing depends on the second, and agentic test management covers where agents fit around both.

KPIs That Reward Judgement

Do not lead with duplicate count. It rewards deleting. A cleanup that removes 20% of the repository and three real variants with it made the suite worse and the dashboard better.

Treat defect leakage as a guardrail, not a benefit. Deleting duplicates does not prevent bugs. It frees reviewer time, and quality improves only if that time goes into uncovered risk.

A Rollout Strategy That Starts Small

The classic mistake is scanning everything at a low threshold and deleting whatever comes back. Do it in this order instead.

• Snapshot first - export the repository and write down what must never be deleted.
• Enrich thin cases - add steps and descriptions to title-only cases in the folders you will scan first.
• Pilot one folder at 80 - compare every candidate and log each decision. You come out with your own precision by band.
• Pick the best base - promote the strongest case in each group, and move anything worth keeping into it before removal.
• Delete copies, archive overlaps - delete only confirmed copies. Archive anything you are unsure about.
• Widen slowly - go folder by folder, and drop into the 50 to 80% band only once precision holds.
• Turn on prevention, then rescan - with Smart Context and ID-matched imports in place, the next scan should return fewer candidates. If it does not, find where the inflow comes from.

The best moment to run this is right after a migration. Merging two legacy repositories exposes overlap neither team could see. Scan before the merged project becomes the working source of truth. One-click migration from qTest keeps folders, custom fields, attachments and linked Jira issues, and leaves the source read-only, so you can scan without touching the old system.

Settle with security first

• Platform posture - TestMu AI security documentation covers SOC 2 Type 2, GDPR and CCPA compliance, TLS in transit, and production kept separate from testing and staging.
• Embeddings - ask how embeddings are stored and retained as part of your DPA review.
• Secrets - keep credentials, personal data and card numbers out of test steps. An organisation-level generation instruction can enforce this.
• Retention - choose between archive and export before the first delete.

To keep a big repository reviewable in the first place, see managing test cases in large projects, and test management for how this fits the wider process.

Try It on Your Own Data

Start with data you already have.

• Bring a CSV - export test cases from your current tool or spreadsheet, bring them in with CSV import, and run a duplicate scan in Test Manager. It costs nothing if it finds no duplicates.
• Migrate a full project - move it from TestRail, Xray, Zephyr Scale or qTest with one-click migration, then scan it before your team starts running it.

Record two numbers from the first pass: candidates found and candidates confirmed. That is your baseline, and it beats any published threshold. For a screen-by-screen walkthrough of the scan, the review queue and archiving, see test deduplication in Test Manager.