When Software Starts Thinking [Testμ 2026]

One Question, Two Answers

The opening claim is first-person: his team asked their system whether a contract clause would hold up in a cross-border dispute. The system is never named, dated or sized.

Both answers appear as slide text. One says the clause is enforceable under the governing law selected; the other says it likely fails because the selection is ambiguous.

His framing line is the session in one sentence: this is the same question being asked of the same system. No screenshot, log, transcript or timestamp of the actual exchange is shown.

It is the strongest illustration in the deck and also the least evidenced, presented as something that happened with nothing to corroborate it.

He also misremembers it later, attributing the contract example to the translator. They are two separate illustrations, and the conflation is noted here rather than repeated.

The Deterministic World

He rebuilds the old model from a deliberately mundane example: a Word document, a typed paragraph, a print command, a printed page. Every time the action runs, the output is the same.

Location-independence is part of what he means by determinism. It does not matter whether you are in India, the US or Europe; the output is identical every time.

The testing consequence follows directly. Any time the output changes, a tester sees an issue, and an issue means logging a defect.

He walks the whole defect loop out loud, from logging through assignment, fix, retest and closure, to show that the entire ritual presupposes determinism. That, he says, is the premise quality engineers were brought up in: you pass a test case or you fail one.

He does not declare that model dead, which is worth stating because the published description does. His position is that it remains right for deterministic systems and that AI is a different class.

The Translator Turn

The pivot example is a translator. Feed one paragraph in three times and get three responses back, and his verdict is that a case can be made for all three being correct.

He explicitly refuses to call any of them wrong, saying there is nothing he can figure out as wrong in any of the responses.

His diagnosis is interpretation rather than randomness. It is what translators do, and interpretation is what has now entered the software world too.

He generalises the mechanism: these systems are not only generating content but interpreting and then responding on that basis, so the same input yields a different answer every time.

The translator becomes his running metaphor and supplies the closing image of the talk, so it carries more weight than a throwaway analogy.

Words Added, Meaning Changed

His hallucination example is a two-line before and after on a slide. The input is a delivery clause promising delivery within 30 days of order confirmation. The output promises delivery within 30 days with a full refund guaranteed for any delay.

His analysis stays small-scale on purpose. The input was a single statement of six or seven words, words got added, and the meaning changed.

He then pushes past the usual version of that point, adding that sometimes the answer is exactly what you anticipated and is still wrong.

The surface characteristics are what make it hard to catch: a response that is confident, well formatted and grammatically correct.

Nothing here is measured or demonstrated. It is an illustration of a failure mode rather than data about one.

Drift Down An Agent Chain

The drift example is a three-agent chain on a slide. Agent A receives an instruction to ship an assignment by Friday, agent B reads that as shipping with high priority, and agent C emits a more urgent instruction again.

The third agent’s output is the punchline of the example and is unrecoverable from the captions, so the escalation is described here rather than quoted.

His point is that no single agent malfunctioned. The meaning degraded across handoffs, arriving at the third agent as something materially different from what left the first.

He names it as its own failure class, and immediately guards against treating the two he has covered as the whole set. Hallucination is one failure mode, drift is another, and both are just two.

The further modes he lists get one clause each and no elaboration: bias, inaccuracy, compromised safety, and exposure of personal or health information. None receives a test, a metric or an example.

Trustable, Not Correct

The vocabulary change is explicit and he flags it himself. He is using trustable and no longer using correct or incorrect, and he names that as the central shift in the discipline.

The contrast he draws is between what determinism gave you, meaning accurate or inaccurate, pass or fail, right or wrong, and what AI gives you, which is a question of overall trust.

His reframing of the tester’s job follows. Answers can all be correct and can all be incorrect, so the work becomes figuring out how close you are to the correctness you are seeking.

Later he restates the target question: it was never whether the system can answer, but whether the answer you are looking at can be trusted.

He asserts twice that QualiZeal has cracked the operationalisation of this. Nothing is shown to support either claim.

Trust Attributes And NIST

He walks three attributes conversationally. Reliability asks whether the system consistently gives an accurate response. Security asks whether someone could slip a hidden instruction into the AI. Human oversight asks whether there is enough human involvement, with a human eye on the output.

He then credits the National Institute of Standards and Technology, saying its AI risk management framework calls out six trust attributes. That count is doubtful. The framework’s characteristics of trustworthy AI are normally enumerated as seven, and the published chapter list repeats his six as though it were established.

His own list, read aloud in the same passage, also has seven items: reliability, safety, explainability, security, privacy, faithfulness and human control. That is consistent with his later remark about six other attributes and inconsistent with the six he attributes to NIST.

The gating rule he states is absolute. All of the attributes need to pass, and if even one fails the system is not trustable.

That absolute sits in tension with two later positions in the same session, a composite trust score above 95% as the release gate, and per-attribute weighting introduced in the Q&A. The three models are never reconciled.

No attribute is defined formally, no scoring rubric appears, and the seven-item list is never mapped onto the six he cites.

The RMTE Loop

He introduces it as four steps within the framework and names the acronym only once, late in the talk. The expansion is never spoken and follows from the four steps themselves.

• Risk. If something can go wrong, that is a risk. For every trust attribute, enumerate how it can fail. Reliability’s failure mode is hallucination; a retrieval system’s is citation sufficiency. He is candid that this step is unfinished industry work that many people are still researching.
• Metric. How the risk gets measured, which he concedes is the hard part. His worked answer for hallucination is a rate: the number of times the system hallucinated over the number of times it was asked to respond.
• Test. Running enough questions against a ground-truth set. As soon as the system returns something not found in the ground truth, that is the signal it has started hallucinating. The published chapters label this step four; in the talk it is third.
• Evidence. Packaging the chain as an audit trail, which is how he says a fully traceable system gets built. No sample evidence pack, report or audit artifact is shown.

He says QualiZeal has defined around 170 metrics and is still adding. No source, sample, baseline, method or published catalogue accompanies that number, and beyond four metrics none is named.

Thresholds And The Full Gate

His analogy for a threshold is the release gate every tester already writes. A test strategy saying a 95% pass rate in UAT clears the system for production is a threshold somebody chose.

The worked hallucination gate is five hallucinated responses in 100 shipping, and six in 100 going back to the developer for a fix and a retest. Seconds later he sets the reopening condition as the rate going below 5%, which would reject the case he had just approved. He never corrects it, and the contradiction stands.

He rejects zero tolerance as unrealistic, saying hallucination is by design and cannot be stopped, and rejects perfect trust outright as utopian because you cannot test everything.

Then he stages the trap. One trust attribute, three or four risks under it, ten metrics, every metric past its threshold. Is that enough to ship? It is not, because it is one attribute and six others remain.

The full gate is every metric, on every risk, on every trust attribute, above its threshold. He is explicit about the cost, saying this is where the complexity and volume of testing really increases, and he assigns the sign-off personally to the tester who certifies the system as trustable.

The composite version is a final trust score above 95% clearing production. That 95% is illustrative, with no client baseline, no derivation and no method for computing the composite. The session offers no sizing anywhere either: no test count, run duration, cost, team size or client engagement.

Citation Sufficiency

He reframes the tester’s task as fact-checking rather than proofreading, where proofreading was the old model of scanning for errors and handing them back to the author.

The retrieval question he poses is how to make sure a system fetching from your repository is fetching from that repository only, rather than creating information.

Citation sufficiency measures how many responses carry a citation at all. His illustration is 95 of 100 responses cited and five not, giving 95% sufficiency.

Citation accuracy is the separate question of whether the citation points at the right paragraph and line, or merely at something that exists in the document but does not answer what was asked.

His conclusion is that proving a citation came from the cited document is not enough; you also have to prove it is the right citation.

How citation accuracy is actually measured, by whom, against what and at what sampling rate, goes unaddressed. No retrieval pipeline, retriever, index or evaluation harness appears on screen.

The Line Into Fiction

He returns to the translator to close, noting that translators are allowed some creative licence and can get creative.

The line that lands the argument is that with too much creativity, what appears on your screen is no longer translation but fiction.

The problem he hands to the audience is that nobody has yet decided where that thin line sits for AI, beyond which its responses become fiction.

He frames it as genuinely open rather than solved, saying it is a question everyone needs to ponder and that many already are.

The talk ends at 29:11, and the recording then carries roughly 25 seconds of dead air before he asks whether the host is on mute. That gap is a microphone failure rather than a pause for effect.

Q & A Session

The host read questions out of the Q&A box, naming each questioner on air. None is a public figure, so they are attributed to attendees here.

• How do you validate an agent’s tool selection before it executes actions against production?

  Chinmay Kothari: There are metrics for tool selection, measuring how often the agent calls a particular tool and whether it was the right one, gated on a threshold, say 95%, or 100 correct selections out of 100 tests. The threshold itself is the open question, and I would put ownership with a risk manager or a new role yet to be defined. He concedes the threshold rather than naming one, and defers who owns it. This is a partial answer. The pre-production half of the question goes unanswered: no sandbox, dry run, staged rollout, permission scoping or human approval is offered.

• How much human intervention does an AI-generated test need before it can be trusted in production?

  Chinmay Kothari: The quantity is difficult to answer. What governs it is the system archetype: a system handling no personal or health data earns a higher tolerance for going wrong, while one that does makes even a single occurrence unacceptable, and an internal chatbot gets more latitude than a customer-facing one. That is an explicit non-answer on the quantity, a redirect to system archetype. The question asked about human intervention and the answer is largely about risk tolerance and test volume.

• If we meet again next year, what will have changed the most?

  Chinmay Kothari: I hope the theory holds up by then. The field is fast-moving, and QualiZeal is building toward testing every kind of AI system, with a product that takes your information, does the testing and returns a trust score, and that is what we will be selling. This is largely a non-answer, opening on a self-deprecating joke, and its one concrete prediction is a roadmap aspiration rather than a shipped capability.

• How do you test an agent’s decision-making process and not just its final response?

  Chinmay Kothari: An agent has layers and every layer needs testing: for a retrieval-based agent, whether the retriever fetches the right information, whether the right context gets built, and whether that context renders accurately in the response. Each layer is tested for every trust attribute and every metric defined for the system. End-to-end-only testing will not do. A test strategy should start from the architecture, enumerate the layers in detail, and test each, because only when every layer is trustable is the final system trustable. This is the strongest answer of the session.

• Are confidence and correctness together the most effective combination?

  Chinmay Kothari: It depends. For a calculation system, yes. For a system filtering resumes for a talent acquisition team, I would care more about inherent bias, and whether results favour one demographic over another. Naming any single attribute as the most important will not hold. The framework lets a user set weights per trust attribute, faithfulness above accuracy for resume filtering and the reverse for calculation. That is a qualified it depends, and it is also the only mention of weighting in the session. No weighting mechanism or worked example is shown, and it sits in tension with his earlier rule that one failing attribute makes the system untrustworthy.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.