Agentic Engineering and Quality in Healthcare [Testμ 2026]

Where Do Agents Actually Help?

Chad Ullman places his organisation’s posture honestly. Being in healthcare and in insurance, they are naturally very conservative, so they did not jump on AI the way more adventurous companies did, though he is careful that conservative does not mean they did nothing.

His concrete win is work they would previously have sent to a consultancy or a partner, at a cost he hedges as maybe 250 thousand dollars, done instead by augmenting existing teams with agentic AI. That is the origin of the line the moderator immediately says he will steal.

Mark Kroh names two places it is clearly working: test authoring in the IDE, where an engineer describes intent and gets a runnable test instead of handwriting selectors, and automated first-pass code review, so human attention is not spent on the boring category of problems. Both are real gains and both save real hours.

Rohit Jaiswal frames the gain as amplification rather than substitution. AI absorbs the mundane task of writing test cases while the tester keeps the domain knowledge and the context.

His specific example is traceability rather than volume. Because AI traces generated test cases back to the requirement, the question of what the test coverage of your application actually is, which he says was very hard on a large application evolved over years, becomes answerable and maintainable. He attaches a condition he repeats all session: it works as long as you keep an eye on it, actively review the artifacts and keep the context tight with guardrails.

The Coverage Metric Broke

Mark Kroh’s most useful observation is a side effect rather than a benefit. Cheap test generation broke one of his metrics, because coverage used to cost something and therefore carried information. When he had 80% it meant somebody worked for it, and now an agent gets there before lunch.

The number survived and the signal did not. What did not get cheaper is deciding what the right answer is, the assertion, which is still hard and is now the entire job. His bottleneck moved from writing tests to knowing whether the tests check anything.

He then offers what he calls a result worth knowing. Take an LLM, have it write a test suite, then ask it to make that suite better repeatedly. Every round it adds more tests. Measured by mutation score, which means breaking the code in hundreds of small ways on purpose and counting how many breaks the suite notices, the self-improving version plateaus around 53%. More tests every round, no more bugs caught.

Change one thing and the picture inverts. Instead of asking the model to improve the suite in the abstract, show it the specific breaks that got past it, and with the same model and the same code the mutation score rises to 90%.

Worth stating plainly: he never says whose experiment this is, and names no model, codebase, sample size or source, so it is a result he passes on rather than one this panel produced.

His generalisation is the takeaway. LLMs are not short on capability, they are short on feedback, and they have no way of knowing what they are not checking.

Sign Off On The Protocol

You cannot sign off on an artifact that adapts, so you sign off on the protocol instead: here are the changes we anticipate, here is how each one gets validated before it takes effect, here are the acceptance criteria written down in advance, and here is the risk assessment if we are wrong about any of it.

He points out the FDA formalised that shape for AI-enabled devices as the predetermined change control plan, and disclaims it in the same breath, on record: his employer is not a device manufacturer and does not file them, so he is borrowing the framework rather than claiming the experience. That disclaimer belongs with the idea wherever it travels.

His argument for it is in the name. It is not permission to drift, it is permission to run a protocol you already validated.

Operationally, the eval suite becomes the regression suite, and exact match assertions have to go, because you are testing something probabilistic. You assert over a distribution with tolerances and treat that layer as a measurement instrument rather than as the function.

His warning is the part he says is under-discussed. The thing that made your system reproducible was pinning the model version, and that is precisely what a vendor deprecation invalidates. When a provider updates the model behind an alias endpoint it keeps returning 200s while the output becomes something you never validated, with no signal that anything changed. Saying you pinned the model is not a validation strategy, it is a dependency with a clock on it.

Continuous Auditing

Rohit Jaiswal describes the shift he has watched over the last couple of years. Testing used to certify at the end and hand the system to operations in maintenance mode, but where the same question asked twice has a high probability of a different answer, testing has to be continuous.

What he now sees across the agentic solutions his teams build is auditing, and he frames it as trust-building rather than compliance theatre. It used to be one and done, and now you have to establish that trust, and it will not be perfect on day one.

He grounds it in what his employer is building, which includes voice agents, chatbots and solutions that generate test artifacts. Since his employer is disclosed on air as a strategic partner of the conference host, treat those as vendor descriptions rather than as evidence; no metrics, customers or outcomes accompany them.

His stated reason auditing survives when sign-off does not is traceability. Whenever something fails it needs to be traceable to where exactly it failed, and he says teams have started asking about auditability and responsible AI as the first question on any proposed agentic solution.

Chad Ullman adds the requirements-side corollary. Whoever is responsible for ongoing validation needs to understand what the agent is doing, what the contract is and what the intentions are, which sends him back to an old term: figuring out our requirements. His framing of the model is deliberately unglamorous and is the panel’s most conservative note, that it is a very smart tool and still a tool.

What Does Safe To Ship Mean?

Rohit Jaiswal reaches for a claims example: an 837 file where a claim submitted through two clearinghouses produces a duplicate payment. He says that class of issue used to take a couple of months to surface in production, and that fast AI-assisted data comparison plus in-system monitoring brings the feedback back far sooner. No organisation, dataset or timeframe accompanies the claim.

His argument for why agents change the economics of monitoring is blunt: an agent is never tired and works at the same speed at eight in the morning or five in the evening.

He extends it to prior authorization, saying several large payers are adding AI components to speed it up because of a CMS mandate tied to star ratings, and that CMS is tightening interoperability rules. He cites no rule, docket or effective date, and he names specific payers who had no representative on the panel, so this is his characterisation of the regulatory position rather than a sourced account. It is also worth noting that one of the payers he names is a joint-venture parent of another panelist’s employer, which goes unremarked in the exchange.

His structural claim is that testing has not died but distributed across the pipeline, so everybody now owns quality. He starts to say testing is dying and corrects himself immediately, and his provocative version is that there are no testers as such, only auditors, which he flags as his own coinage.

He also names the cost of that distribution. The old fallback was that testers would catch the bugs, and now if what a tester generates with AI is wrong in the first place, it trickles down and becomes very hard to trace. Debugging means understanding what the AI did and where exactly it is breaking.

Optimize For Reversibility

Mark Kroh answers the same question from an engineering-director rather than a testing mindset, and reframes it. There are plenty of operational gates for keeping bugs out of production, and the question he now asks is how fast he can un-ship a release.

His justification is the panel’s cleanest argument against prevention alone. Gates are only as good as your imagination about what could break, and agents are writing code faster than that imagination is scaling, so you are going to miss things.

His concrete practice is one flag per flow, evaluated in one place, with a removal date, and he immediately names the failure mode: accumulating a hundred flags and losing track of which ones mean anything.

Mobile is what disciplines the model, because there is no instant rollback there, so that is deliberately where his gate is heaviest. The asymmetry should be applied everywhere: heavy gates where reversal is slow, light gates where you can undo it in a minute.

His diagnosis of the common failure is that most organisations run gating uniformly, which means it is simultaneously too slow in one place and too dangerous in another.

Chad Ullman endorses it and says the crisp sign-off moment has essentially gone. You no longer reach a point where you can say it is safe to ship; you say it has passed every gate and you are ready to try, having engineered the system so that when it goes south you can reverse quickly. That is worth holding against the way this session was billed, since the panel argues almost the opposite of a zero-tolerance-for-failure posture.

Data Mining Beats Synthetic

Asked how they avoid a false sense of security from synthetic data, Chad Ullman describes a reversal. Last year his team ran a large evaluation to move toward more synthetic data, having relied for a long time on de-identified production data after a major de-identification programme.

AI caused the reversal rather than policy. They realised they did not have to rely on synthetic data, because they have a massive real-world de-identified data set and can now query it far more effectively than before.

His worked example is a claims scenario: find an enrollee with a very specific claims history, such as a claim denied because they have had too many of a certain procedure in a year. Previously that meant a lot of SQL and manual searching, and now you describe the scenario and get the data back.

His conclusion is a named strategy shift rather than a tool choice: move away from synthetic data to data mining, which is how his team is approaching it and what is working for them.

Mark Kroh agrees and adds why teams do not do it. De-identified production data obtained through HIPAA’s expert determination path is genuinely messy, and teams often skip it because synthetic feels safer.

Synthetic Data Deletes Tails

Mark Kroh states the structural objection in one sentence: the very mechanism that makes synthetic data private is the mechanism that deletes the edge cases. Generators model typical records well, and the tails of the distribution, meaning outliers and complex comorbidities, get erased. He attributes this to research without naming a study, author or date, and it is a clinically consequential claim, because the erased tail is patient populations.

His first prescription is to stop measuring the wrong thing. Fidelity scores only prove your synthetic data matches the aggregate distribution, so measure the gap between training on synthetic and testing on real data, and gate releases on that delta.

His second is to generate the ugly cases deliberately, and he names a failure mode he calls vintage. Every synthetic patient is perfectly self-consistent and born today, while real patients have accumulated state across years of migrations, abandoned flows and schema changes.

He backs it with a first-hand failure: they shipped a release that broke production because the synthetic test could not simulate a multi-year-old account. Test data has no vintage, so you have to build it.

His third point closes the loop back to his own theme. Accept that you cannot test your way to edge case coverage, and instead close the gap fast with detection in production, which returns him to optimising for reversibility.

Edge Cases From Tickets

Rohit Jaiswal agrees that de-identified production data is the typical go-to in every large enterprise, and says the hardest part was never obtaining it. It was maintaining it, and mimicking the edge cases.

His team’s answer inverts the usual target. Instead of following the happy path, they target the ugly edge cases and the defects arriving in production, feeding a year’s worth of production incident tickets into a generator to produce data mimicking those negative scenarios.

He is clear about which half is hard. Creating synthetic data to mimic the happy path is table stakes, while having the insight to create data supporting edge cases or anomalies is where teams struggle.

His maintenance warning is specific. After de-identifying, the same data gets used by many people across the enterprise, and after three or six months it goes stale. Then you do the data refresh and suddenly all your test automation starts failing. That is an experience report rather than a measured pattern.

His claim for AI here is repair speed rather than prevention, since those refresh-induced failures can now be fixed fast compared with the pre-AI era.

Human On The Loop

Mark Kroh refuses the premise of the question and says so up front. Most of the time he does not see the human in the loop as a meaningful safeguard, and he offers specifics rather than reassurance.

His evidence is a failure inside his own organisation. They turned on AI code review partly because of review fatigue, adding a reviewer that also is not human, and then it stopped working for a stretch because the context they fed it grew past the model’s limit. Every review failed on size, the failure rendered as a small stub that read like a normal status, and nothing escalated. The reviewer added to compensate for overloaded humans was itself not reviewing.

His general rule follows: human review at agent scale degrades into rubber stamping unless you change what the reviewer is inspecting. You cannot focus on the volume, you have to focus on the target.

Two mechanisms follow. Attribution, so that knowing an agent wrote something lets you route it to real scrutiny instead of everything getting the same skim. And signal over pile, because handing a reviewer 200 generated tests gets a thumbs up, while handing them one number saying the suite does not catch anything gives them something to act on.

He then places humans at three deliberate points in his own products: before, where clinical content and AI outputs are reviewed before entering the knowledge base or the regression suite; during, where high-risk intents such as crisis language escalate to a licensed coach; and after, where humans label conversations to keep eval suites tied to reality. These describe safeguards inside his employer’s own consumer health products, with no efficacy data, clinical validation or oversight body named, so they are worth reading as one company’s design rather than as a validated pattern to copy.

His bright line is clear: the loop has to be genuinely real anywhere an agent holds write access to a system of record and can act without approval. Everywhere else, design for the fact that the human is going to skim.

Rohit Jaiswal takes the opposite emphasis, on training rather than architecture. His organisation coaches newcomers and junior testers never to bypass the loop, because skimming every test case initially is what builds the fundamentals and lets you improve the context and guardrails. He worries the next generation will never acquire that if they enter the profession with AI already doing the reading, and he is candid that the work is tedious.

Coder Becomes Communicator

Chad Ullman says the line between developer, tester and automation engineer has been blurring for years, and what AI does is let it disappear completely. The new differentiator is the ability to interpret for the AI, providing intent and context and conveying an understanding of the business and the ecosystem. Instead of a coder, you become a communicator.

His coaching advice is against heads-down work. You cannot take a requirement and just code it, because even when defining tests you have to understand the big picture in order to supply that context, which makes your language important.

Mark Kroh agrees that the work that got cheap was script authoring, and names four priorities that replace it. First is oracle design, deciding what correct means, which he calls the scarce skill now that producing volume is basically free.

Second is eval design, which he insists is a different discipline from test design: sampling, rubrics and statistical thinking, grading distributions instead of asserting equality, which most testers were not trained for.

Third is reading a suite critically, being able to look at 90% coverage and say you do not believe it and then explain why, a skill he says was always the valuable one and used to be buried under a pile of typing. Fourth, and in his words most underrated, is test data generation as a real engineering discipline, deliberately constructing the ugly cases a generator will not produce.

His reassurance to worried automation engineers is the panel’s best line on careers: the part of your job a model does well was never the part that made you good at it. Rohit Jaiswal adds the shortest version, that the real moat now is business domain understanding.

Mechanisms Versus Sentences

Asked for one strategic objective to take back, Chad Ullman declines to give a strategy and gives a stance. AI is a super suit, and it is only as good as the person you put it around, so do not forget to invest in your people as much as in your technology.

Rohit Jaiswal’s is a four-word instruction with a prediction attached: adopt the agents, but own the context. The winners in healthcare will not be the best models but the teams with the best domain knowledge feeding them.

A sentence, in his definition, is a rule that lives somewhere an agent can read it: a prompt, a context file, a message announcing a code freeze. The agent will read it, agree with it, and can do the opposite.

His example is the Replit incident, where he says an agent dropped a production database during an active code freeze. The detail he says matters is not that the agent went rogue: the freeze existed, the agent knew about it and agreed to it, but the freeze was text rather than a permission. He recounts it from memory, so it is his account of a third-party incident.

The second route to a sentence is the one he worries about more, and it ties back to his own failure earlier in the session: a control that genuinely used to be a mechanism and stopped running without anyone noticing. A reviewer was assigned, the check was green, the paper trail was perfect, and the safeguard was gone. Both feel identical from the inside, because you think you have a control and you do not.

His pricing argument for the urgency is the session’s closing idea. Agents did not invent new risks, they repriced the ones you already had, because a broken gate used to leak a handful of unverified changes a week and can now leak hundreds. Hence the one action: verify that every required check in your pipeline actually executes against everything it claims to cover, not that it passes but that it runs.

Q & A Session

The chat supplied both closing questions.

• What issues do AI agents and their output typically encounter when interacting with HIPAA compliance?

  Mark Kroh: This is not a quick question. There are two vectors. First, what data you give your models access to, which means making sure the environment itself is locked down and compliant. Second, running your evals thoroughly enough that things are gated, so the models are not providing unauthorised advice and have the proper breakouts, such as escalation when somebody presents suicidal ideation. It comes down to tightness around the entire process.

  Chad Ullman: You sometimes have to think beyond HIPAA compliance to the contractual language with the groups you work with. One organisation we work with is very wary of AI, and that creates obligations which complicate things. A follow-on remark about developing an algorithmic alternative for clients who say they do not trust AI is not clearly attributed in the recording, so it is left unattributed here. Rohit Jaiswal declined to add anything.

• If AI can generate software faster than organisations can understand it, are we creating an accountability gap?

  Rohit Jaiswal: Testing can no longer sit at the tail end. It has to start from the start, which is the shift-left conversation we have all had with varying maturity. It connects to auditing, because the audit trail people now ask for is the test artifact. Run these agents inside an old-fashioned lifecycle and you leave the gap in place, create a bottleneck and will not get a sustained result. His is the only substantive answer here, and Mark Kroh did not answer this question.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.