AI-Powered Impact-Based Testing for Faster Safer Releases [Testμ 2026]

Ten Lines, Fifty Thousand Tests

The opening scenario is a slide hypothetical rather than a measured UKG incident. A developer changes ten lines in one service, nothing unusual, and the commit puts hundreds or thousands of test cases into the queue with a feedback delay of four hours or more.

Navneet Goyal reframes twice. The question is not whether all the tests can run faster, but whether they need to run at all, and then whether 50,000 queued tests share the same probability of being impacted by those ten lines. The 50,000 is part of the hypothetical.

His decade framing sets up the mismatch. Code changes got smaller, builds got more frequent and deployments got more frequent, and how teams run their test cases did not change.

Compute is the cost he treats as easy, because everyone can see the infrastructure, environments, databases and runners behind each regression run.

Developer context switching is the one he says teams do not count. The developer does not sit idle waiting, moves to another story, and hours later has to switch back, reload the original problem and reframe the solution.

Pipeline congestion follows as a second-order effect, with parallel pipelines and integration workflows delaying deployment. His conclusion is that all of it adds up to a developer productivity problem.

The Regression Paradox

The name is his own coinage, presented on a two-column slide with the choices at either extreme.

One side runs everything: maximum coverage and the highest confidence, paid for in slower feedback, infrastructure cost and a hit to developer productivity.

The other runs fewer tests: faster feedback and lower cost, with no way to explain where the confidence comes from or to justify the selection.

His resolution is not a smaller number but a defensible one. Rather than blindly running a few tests, pick the right tests so confidence is not damaged, and make that confidence something that can be explained and justified.

Explainability as the price of running less is the sturdiest claim in the session, and it is what the second half is engineered to deliver.

Four Tactics That Break

He is explicit that none of this is new, noting that teams have been doing test selection for many years.

Four existing approaches come off a slide: tags and priorities such as P0 and S0; a static mapping between test cases and components; path rules tying particular files and paths to particular tests; and manual ownership, where a team owns a set of tests.

His objections are practical rather than theoretical. Tags need discipline, because every new test has to be tagged correctly. Component mappings go stale as tests get added and edited. Manual ownership works in a small organisation and becomes a challenge across large distributed teams.

What all four share, in his diagnosis, is a missing ingredient: understanding. Without knowing what code change entered the system, selecting tests is guesswork.

The specific blind spot he names is the non-obvious one. If file A imports file B and file A changes, that correlation is visible, and there are many downstream dependencies that reading the code will not reveal.

Flip The Coin

The pivot is a change of starting point. Reasoning about what to run by looking at the test suite does not work, so flip the coin and look at the code change first.

He frames it as moving from regression to intelligent regression, which is the cleanest one-line summary of the session’s argument.

The pipeline on the slide has four steps: look at the code, identify the direct and indirect impact, rank the resulting tests by risk, and execute that subset for faster confidence.

The question the strategy asks of every commit is which tests are most likely to be affected by this exact change, and that answer set becomes the subset to run.

He asks for a trustable and reliable mechanism for deciding what should influence the include-or-exclude call, which is what sets up the explainability signals in the second half. None of it was demonstrated; this section is a slide walkthrough.

Five Kinds Of Impact

Direct impact is the obvious case, where a change to a file maps to test cases tied to that component, feature or path.

Indirect dependency covers the multi-layer downstream relationships nobody has visibility into.

Semantic relationship covers components or paths working on the same behaviour, related by what they do rather than by an import edge.

Historical relationship is the pattern evidence: whenever this component changes, these particular tests usually get impacted, fail, or start behaving differently.

Critical path weights the component rather than the change. A small change to something like payments or authentication deserves more regression than an identical change to a less important component. Those two are offered as examples rather than as UKG systems.

Combining all five signal types is what he calls the route to impact-based testing, and it is the bridge into the implementation half.

Six Signal Categories

Srikanta Sahoo states the core idea plainly: instead of running the complete regression suite, identify the tests most likely to be impacted by a specific code change.

More than 20 signals feed that decision. Counting what he then enumerates gives roughly 22, so the claim holds up against his own list.

His governance caveat lands before any of the mechanics, and the captions garble it badly, so it is paraphrased: the AI recommends the tests, and engineering policy and risk controls still govern what finally executes.

The algorithm has a name he expands as unified impact fusion algorithm. The acronym itself is unintelligible in the recording, so it is described here rather than printed.

The six categories come through cleanly: semantic, structural, historical, quality, dynamic and the machine learning part.

He also gives the decision order. Structural comes first, asking what is actually connected, then semantic asking what is related, then historical asking what has happened before, then quality asking whether it can be trusted, and finally an analysis of the change’s risk and urgency before the impacted tests get ranked.

Structural And Semantic

The structural signals on the slide are dependency analysis, which identifies which modules, services or components depend on the changed code and reveals both direct and transitive paths; import analysis, which finds files that directly import, reference or consume the changed class, module or function; call graph analysis, which traces which execution paths may reach the change; package coupling; and file proximity within the same folder, package or module.

He misfiles package coupling mid-sentence and corrects himself, which is a slip rather than a change to the taxonomy.

His summary of the category is that structural signals say which parts of the system are technically connected to the change.

The semantic signals are semantic similarity, comparing the business and functional meaning of the change against the intent of candidate tests using embeddings or a language model; code similarity; path similarity; and metadata matching across product, component, feature, repository tags, labels, platform, workflow and test type.

His gloss on embeddings is delivered live rather than read, and is the clearest thing in the section. Search for the word bank and you get both the place you withdraw money and the side of a river; the meaning has to come from context.

The category summary follows: semantic signals say what the change is about rather than where it sits.

History, Quality, Dynamics

The historical signals are co-change history, learning which code and tests change together across prior commits and pull requests; failure history, which looks at what previously failed for similar changes while separating real defects from flaky or infrastructure failures; test recency, weighting fresher evidence more strongly through time decay rather than hard exclusion; and test frequency.

The quality signals are a trust score combining supporting evidence into an overall confidence level; coverage overlap, checking the selected tests collectively cover the impacted area without excessive redundancy between near-duplicates; explainability, which gives a reason for each selection such as dependency match, semantic relevance, historical failure or component alignment; and adoption metrics drawn from direct developer feedback.

The dynamic signals are change type, risk score and priority score. Change type distinguishes bug fixes, features, refactors, configuration changes, dependency upgrades, database changes, API contract changes and test-only changes, on the reasoning that each warrants a different breadth of testing.

The risk score is a blast-radius estimate built from the files changed, whether a critical component was touched, dependency depth, complexity, production sensitivity and historical defect patterns, with higher risk broadening the selection.

Two machine learning models sit behind the ranking, one updated on every execution and the other retrained on a fortnightly cadence, chosen because one suits linear data and the other non-linear. Both model names, along with the embedding model, are unclear in the captions and are not printed here.

The Scoring Example

The scenario on the slide is a change to password length validation inside an authentication service, which he says would traditionally trigger a thousand authentication regression tests. It is presented as a real-world example, with nothing indicating it was an actual incident rather than an illustration.

The engine instead takes direct dependency, semantic relationship, previous failures and test reliability, ranks the candidates, and selects those carrying the strongest evidence of impact.

One scored test appears on the next slide: semantic similarity 0.92, dependency match 1.0, historical match 0.85, trust score 0.90, machine learning ranking 0.88, combining to a final impact score of 0.91 on a scale from zero to one.

The weights that produce that combination are never shown or stated. A reader cannot reproduce 0.91 from the five component scores, which is worth saying plainly rather than implying a published formula.

The threshold slide has four tests at 0.94, 0.82, 0.77 and 0.12. The team sets the threshold, and at 0.50 the first three get selected while the fourth does not. He misspeaks once in that sentence, saying the threshold rather than the score is below the threshold; the arithmetic is consistent even where the wording is not.

Architecture And Learning Loop

The integration principle is one of the strongest practical points in the talk. The system does not replace the existing CI/CD ecosystem; Jenkins, GitHub Actions or GitLab keep triggering the pipeline, and before test execution the pipeline calls an impact analysis service.

That service returns the ranked impacted tests along with a confidence value and an explanation, which he argues makes adoption easier because a team can add the intelligence without redesigning its delivery pipeline.

The stack on the slide is a Python web API at the application layer, an embedding model for semantic similarity, two gradient-boosting and stochastic-gradient models for ranking, a relational database with vector support for structural data and embeddings, and an in-memory cache for fast access. Every product name in that list is unclear in the captions, so the components are described by role.

His own caveat on the stack is worth carrying, because it separates a reference architecture from a product pitch: the technologies are implementation choices and the architecture itself is technology independent.

The learning loop cycles through ingesting the outcome, extracting new signals, updating the ranking, predicting future impact, executing tests and updating trust. Structural and semantic signals dominate at first, and historical evidence becomes more valuable over time.

The learning is deliberately constrained. Flaky tests, environmental failures, drift and other noise are accounted for, and a failing test’s pass and fail history is used to judge whether the failure was environmental before a weighted score gets assigned.

The Numbers

The maturity curve on the slide runs from about 85% accuracy in week one, to around 92% past 200 builds, to around 96% after a thousand builds roughly a quarter later, to around 98% at six months on a mature system with plenty of data behind it.

Two minutes later the benefits slide gives test reduction of 90 to 95% and states that the accuracy already discussed is 95%, which matches none of the four figures just given. Both readings are reported here because the session never reconciles them, and accuracy is never defined.

API latency is given as sub-two-second, attributed to the in-memory cache, with no percentile, load or payload size.

Scale is given as more than 100,000 test cases in the codebase. That is a statement about codebase size rather than a described validation protocol.

Recall appears on the benefits slide as a dimension and never as a number. The published chapter title claiming recall was maintained has nothing behind it in the recording.

None of these figures carries a source, sample size, baseline suite, measurement window or method. All are self-reported by the team that built the system, read off slides, with no artifact on screen.

Layered Safety

Srikanta Sahoo puts the objection before the answer, asking what happens if the AI skips a test that would have caught a defect, and calling it the most important production question.

His answer is layered safety. Mandatory smoke and critical-path tests run regardless of the recommendation. For risky or uncertain changes the threshold drops automatically to widen the selected set. And full regression remains available as a fallback.

The most useful war story in the session concerns skipped builds. If build 200’s impact set was computed but the pipeline never ran it, the earlier engine only computed build 201’s diff, so build 200’s changes went untested. The current engine detects that build 200 did not run and returns the impact of both builds in the next run.

Brand-new tests were the second lesson. With no history behind them they were not being selected, so a new test is now auto-detected, stored, and added to the next impact set by default, which he notes also exercises the new test’s own stability.

Failed tests get re-added on the next build with a distinction drawn between functional failures and flaky or environmental ones. The passage stating which category is auto-re-added is too garbled to report, so only the distinction is recorded here.

Q & A Session

The host relayed four questions. Two questioners were named from chat; those names are unclear in the captions and are withheld.

• If AI can predict where a change will have the biggest impact, should we still run every test for every release?

  Navneet Goyal: No. But there is always a fallback. Upgrade-class changes such as a Java 17 to Java 21 move deserve full regression. And as a standing guard rail, we tell our own internal teams at UKG that every third or fourth or every nth run should be a full run precisely because it is AI. I want a human in the loop because the system can produce false positives or false results, and I would estimate large changes at under 5% of builds, with 95% carrying four or five commits. He opens with no and immediately qualifies it into something more conservative than the session’s framing, and those proportions are self-reported with no data behind them.

• Should every iteration be deployed, and is there guidance on deployment impact size?

  Navneet Goyal: Does this refer to build deployment? The tool does not govern deployment size, and simply works from whatever reaches the build. As for who deploys every iteration, an enterprise application organisation is one case and a consumer technology company deploying hundreds of builds daily is another. Partly answered, partly deflected. He checks the question first, then pivots to that contrast, and the consumer technology company he names there is second-hand and unsourced. Deployment-size guidance never arrives.

• How do you generate test data customised to enterprise applications?

  Srikanta Sahoo: We do not cater for test data or test case generation. The tests are written by the team, and the engine picks from those to make its predictions. Navneet Goyal closes it by confirming the tool generates nothing and creates no new test cases, working only from what is already in the regression suite. A non-answer, and correctly so, because the question sits outside the tool’s scope. The scope limit is genuinely useful; the question was not answered.

• What happens when an agent finds a faster route that bypasses an important business control?

  Srikanta Sahoo: We retain full control to run the full regression, and we do so whenever the impact set looks inadequate. Navneet Goyal adds the useful part: on day one the results will not all be right, confidence rises toward the six-month figure, and a first quarter or so needs human diligence. A non-answer to the question as asked. The question is about autonomous agents circumventing controls; the answer is about human override of test selection. Neither addresses agent behaviour or business controls.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.