Mission-Critical Priorities in Quality Engineering [Testμ 2026]

Six Bets For The Year

The host set the format up front: every leader placed a bet this year, each had to kill something to fund it, each had to prove it, and each now has evidence. Bet, sacrifice, prove, predict. The panelists went on to dispute the sacrifice half.

Ali Karnib’s bet is transforming from test automation to test infrastructure powered by AI, explicitly not so AI replaces quality engineers or decides releases, but so reliable data surfaces risk areas and engineers can bootstrap their own testing. The role shift he describes is moving quality engineers from being the testers to being the ones enabling engineers to test.

Cem Sever refused the word bet outright, saying test engineers do not like to bet and instead build trust, and that they are now delegating confidence to AI agents. His precondition is blunt: if you are not mature in data strategies and automation, AI will not help as much as you expected. His actual investments are rebuilding the automation framework, a single repository for test cases and automation scripts, keeping compatibility with tools already in use such as Selenium and Playwright, and moving to serverless and event-driven infrastructure.

Akshit Lomash bet on democratising the roles, blurring the line between SDETs and software engineers so that SDETs contribute more to feature development and engineers test their own code more effectively. He calls it aspirational at most organisations he has worked at until AI made it achievable.

Tenny Agustin bet on AI not for its own sake but to take friction out of engineering: making good quality practices easier to apply, identifying risk earlier, and creating more space for engineers to engineer. Hina Saini described her year as adopting AI across the range of skill sets already in her team.

One gap worth recording: Kumar Gaurav Jain was asked for his biggest bet in the same round and gave only a self-introduction before the host moved on, so no opening bet of his appears in the session.

Replaced, Not Deprioritized

The host asked what legacy practice, such as massive manual regression cycles, each leader had deliberately deprioritised to make room for agentic AI. Not one panelist named a practice they had dropped.

Cem Sever answered first and flatly, saying they have not deprioritised the legacy work, but are approaching it cost-consciously, identifying opportunities with real efficiency and treating frequency of reusability as a key factor.

Akshit Lomash openly rephrased the question, saying he does not think they have deprioritised anything and would say they replaced those activities, because deprioritisation carries a connotation of having accepted more risk, and he does not think they have done that.

His replacement is specific. Manual regression cycles that gave confidence the product was of the right quality have been swapped for much smaller cycles with tighter coverage using automation or agentic solutions.

He also listed the repetitive work actually eliminated, being test case authoring, writing automation scripts and maintaining them, and what the freed people now do instead: think about edge cases, learn customer flows, and expand the coverage the agentic solutions provide.

Ali Karnib agreed there is no deprioritisation, just a shift. Late-stage regression cycles are getting smaller, more targeted and more risk-based, while the full automated regression still runs overnight, which bothers no one and still produces a result.

Exploratory Testing Returns

Ali Karnib was the only panelist to name a practice moving in the opposite direction, saying he is seeing a bit more need for exploratory manual testing.

He drew a sharp line around what kind. It is exploratory testing rather than going and checking boxes manually, so the returning practice is destructive exploration and not scripted manual regression.

His reason is a capability boundary rather than nostalgia: AI can bootstrap automation and bootstrap the code, and cannot behave like a human.

The instruction he gives his team is short. This is the feature, this is the risk, go try to break this part manually.

Hina Saini reached the same place independently later in the session, saying her engineers focus more on exploratory testing now that AI handles automation-report evaluation, and that it will always be a niche.

Consolidate Process, Not Tools

The host introduced the tool-sprawl question with an on-air disclosure, noting it hits hard because they are a tool company. TestMu AI is the conference organiser and a vendor in this space, and one panelist later mentions having assessed its product, so both sides of that exchange carry the affiliation.

Kumar Gaurav Jain opened with the bluntest line on it: tool consolidation should not be the end goal, and is absolutely not the end goal. Different teams legitimately use different engineering platforms, and multiple tools exist because they solve very specific problems.

He set a limit in the same breath, saying that does not mean having 30 different platforms or technologies in the organisation. The point is using the right technology properly rather than unlimited choice.

What he standardises instead is reporting, so that no matter which platform, technology or framework is used there is a consolidated place to see overall status, plus a standardised CI/CD everybody follows. His focus is capability and engineering patterns rather than consolidating the tool stack.

His one concession to tool-level consolidation is at the framework layer: build the ecosystem so automation, cross-browser testing, accessibility testing and API testing are all served, so you automate once and the rest follows from the same script.

Tenny Agustin put it as a sequencing claim, that tool consolidation is the secondary benefit of really honing process, because process is where a lot of the burden lies.

Preventing The Wild West

A fellow panelist interrupted with an unscripted follow-up, asking how you prevent this from becoming the wild west, since it can easily sprawl into chaos.

Tenny Agustin conceded the premise first, agreeing there is an element of the wild west to it to begin with. The answer offered is leadership judgement rather than a control, meaning strong leadership making the right calls at the right time.

The two failure modes named are about people rather than tools: investing too much time looking into a tool, and pulling too many people away from their day jobs to do discovery work on tooling.

The mechanism described is a filter role, vetting tools and processes so that what reaches the engineer is not lawless, working against a set of standards alongside other engineering leaders and using proofs of concept and discovery phases.

Hina Saini gave the most concrete guardrail of the segment. TUI has a security team and an AI lab team that issue guidelines, and as a leader in every department there is a responsibility to follow them, with a proof of concept run for any tool before adoption.

Her governing analogy is the manual-to-automation transition: you do not need to automate everything, you need to know what to automate and to have done the risk analysis, and in the same way you do not have to do everything with AI.

The End Of Forced Migrations

Akshit Lomash volunteered the session’s clearest reversal, describing his own prior habit. Until last year, consolidation was one of the first things he would do on joining an organisation, seeing five teams on five frameworks as a chance to consolidate and show quick results.

He named it as a personal update rather than an industry truth, calling it one of the mind shifts he has personally had to make in the agentic world.

The mechanism he credits is abstraction. As an engineer you talk to an agent, and it abstracts out the complexity of multiple frameworks underneath.

He was careful not to overclaim, noting that it does not eliminate the sprawl, it starts hiding it, so the complexity still exists under the agent layer.

The operational consequence is the memorable part. They are getting to a point where they let those frameworks die a natural death instead of forcing expensive migrations, investing in agentic solutions rather than forcing people onto new frameworks. He offers no cost figures for the migrations avoided.

Metrics That Reach Executives

Ali Karnib named two that land: defect escape and incident rate. If those drop, the team is doing its job.

His illustration of why test counts are meaningless is rhetorical rather than measured. He does not care whether it is 10,000 tests or 10 as long as they produce a result, because a thousand tests with a high incident rate means the job is not being done, while ten tests with a very low incident rate means it is.

He ties the urgency of escape metrics directly to AI-accelerated delivery, noting there is vibe coding going on even among engineers.

Tenny Agustin argued from the customer end, since the customer does not care how many tests you write or run or how many pass. The meaningful signal is whether customers have confidence in the platform, meaning few incidents reported and customers not finding edge cases.

Akshit Lomash tracks escape defect ratio as standard, plus trends in severity by product area, which is the view he takes to executives.

He was also the only panelist to defend legacy metrics, and he scoped where: there is space for them when talking to individual teams and contributors, so he still tracks how much test case and script authoring individuals are doing. He is separately pushing into platform observability, covering reliability trends and the blast radius of issues.

Readiness Beyond All Green

Kumar Gaurav Jain deliberately took the readiness question out of the AI frame, saying it has nothing to do with AI and flagging it as his personal opinion.

His central claim is that all test cases executed and passed, with everything green, is not true readiness.

He structures readiness at three levels, technical, operational and organisational, reducing to three questions: can the platform handle the demand, can the customer complete the end-to-end journey, and what is the readiness if anything goes wrong.

Level one is capacity and resilience, meaning whether the platform is capable and resilient enough for the demand.

Level two carries his sharpest line. It does not matter if the platform is 99.99% green, because if the customer cannot complete the journey you are not ready and the product cannot go to production, since it is not solving the customer’s problem. That percentage is an illustrative device rather than a measurement.

Level three is the rollback plan, always having a plan B for how quickly you can recover. He closes on scope of responsibility: you are not the person responsible for test execution, you are the person responsible for ensuring the business runs properly, and it runs properly when customers are satisfied.

The Last Go/No-Go Call

The host pushed past the abstract answers with a deliberately concrete probe, asking what each leader was actually looking at during their last go/no-go call with management.

Hina Saini’s answer was candidly unchanged. They still look at automation results, defect analysis and the results from manual validations, and those have not changed and are not going to.

What has changed at TUI is upstream of that call. Her team has its own prompts and agents doing the analysis of automation reports, and skills that filter release notes and the defects identified during sprints to indicate where defect density is likely and where to focus.

She was explicit that timelines have not moved, saying they have not been able to crunch them further and that she does not foresee achieving that in the next six months or so. The blocker she names is adoption and mindset change rather than tooling, and that candid negative is worth keeping.

Cem Sever’s criteria are no impact to business continuity or to customers and services, no slippage to production, and time to market, plus a new one, since in the AI world the accuracy of the outcome is now a trust factor. He also states his organisation operates at zero defect levels, an absolute claim about a named employer that is not verifiable here.

Paid Off, Fizzled, Promising

Akshit Lomash was the only panelist to sort his AI investments into explicit buckets: paid off, promising, and fizzled out.

Paid off is the replacement of monotonous SDET activities such as test case writing, script writing and test maintenance, where he reports tremendous gains, a velocity increase and productivity improvement. No figures accompany any of it.

Fizzled is test case consolidation. They spent effort consolidating test cases because the work was repetitive, and the results were mixed, because the context part was missing.

The reason he stopped is unusually specific and rarely said out loud: at some point it became a decision between providing more context and the return in terms of token usage.

Promising is coverage expansion against combinatorial explosion. He says the platform is extensible enough to have over 400,000 possible configuration combinations, so they cannot test every customer under every setting. That figure is his own statement with nothing shown to support it.

The approach is to correlate production usage with the combinations worth testing, constantly updating which scenarios to review and add where coverage has gaps, then reuse the already-built authoring and script generation capability to plug them.

Judgment Stays Human

Kumar Gaurav Jain gave the most enthusiastic and the most bounded answer in one breath, saying AI is amazing and he uses it every day, then immediately naming where he is still in wait-and-watch mode.

Where it already pays: creating test cases, automating them, analysing results and setting up test data quickly and efficiently. His summary of the shift is that code has become the commodity.

Where he is waiting is decision-making, since AI cannot take all the decisions on its own, and he would like to see the day it can identify and prepare the right test cases and execute them.

His position on replacement is emphatic and repeated: AI has not reached the point where it can replace human judgment, it cannot replace humans, it should not, he does not want that to happen, and it should be used as a technology that improves efficiency. His verdict rule is that AI has paid for itself when it augments engineers.

Cem Sever split the question differently, separating the testing of AI-developed services, which he says paid for itself from the start and is an easy decision, from AI services used for testing. His cases in the second category are defect segregation and triage including environmental and infrastructural defects, writing test cases from business requirement and technical documents, verifying them, and coverage analysis.

He adds a maturity precondition on the payoff, since the outcome changes according to the maturity level you have in that domain, so investing in lean processes and documentation pays back.

Predictions For 2027

The host asked each panelist for one bold prediction for quality engineering by the end of 2027, explicitly on the record.

Cem Sever expects more human intervention than people anticipate, with skills changing and broader roles such as a chief quality officer emerging, and risk management becoming the hot topic. Hina Saini’s one-liner was that roles will evolve. Cem Sever’s supporting references are two unnamed acquaintances, so they are not citable sources.

Ali Karnib expects quality engineering to own the trust framework, the observability and the behavioural risk model and boundaries, and expects AI to shorten the distance between identifying a risk and acting on it.

Tenny Agustin expects traditional QA to look very different, working hand in hand with AI and getting more sophisticated at anticipating problems intelligently rather than only reacting to or preventing them.

Kumar Gaurav Jain expects productivity and efficiency to improve massively, and adds a career instruction: treat the assignment as a product rather than a project, become the expert of the domain, and gain as deep a knowledge as possible.

Ali Karnib said that is already his hiring bar, since a candidate telling him how many tests they automated is no longer the metric. The host also offered a seventh prediction of his own, that teams shrink to two people running kanban with automation deciding what ships, which is vendor-side speculation rather than panelist testimony.

Q & A Session

Two attendee questions were taken, and a third was read out and never answered.

• What happens when two agents disagree about the correct solution?

  Ali Karnib: This is precisely where human judgement enters, and his answer was about accountability rather than technique. You are the decision maker, and if you are letting AI make that decision for you then there is another problem to discuss, because you are accountable for what the AI produces and the AI is not. He offered no arbitration mechanism, no confidence scoring and no tooling; the whole answer is to use your best judgment. His example involved running two coding agents against each other, and the product names are not recoverable from the recording.

• As agents design and execute tests autonomously, how do leaders ensure speed does not come at the expense of trust and reliability?

  Kumar Gaurav Jain: AI does the job and human intelligence oversees whether it was done correctly. His answer was brief and again about oversight, and pressed lightly he expanded with a transport analogy rather than a process, which leaves the operational question of how to verify agent output at speed unanswered.

  Akshit Lomash: With agentic proliferation the scale of output has grown exponentially, so the key focus for QA leaders should be observability. I see gaps even in my own organisation. It is not important how many tests run, but how fast you can get to the root cause when one finds an issue.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.