Turning OWASP AILLM Risks into Practical QA Checks [Testμ 2026]

The Rewritten OWASP List

The opening claim is dated relative to the talk, not to publication: OWASP rewrote its list of the top ten LLM risks 17 days earlier, and eight of the ten moved position from the previous year. Elsewhere the rewrite is dated to 4 August. No version number, document link or on-screen citation accompanies either.

The rhetorical question that follows is whether anything in your tests changed when that list did. Thejes Sree Satheesh Kumar answers it in the room, assuming the answer is no for nearly everyone precisely because the change is 17 days old, and calls that the reason for giving the talk. The assumption is flagged as one.

Two live polls are run and neither result is reported on air. One asks the room to react; the other asks teams that shipped an AI feature this year how many ran security tests for it, with a request to drop a number in the chat. No number is ever read back.

The answer offered instead is anecdotal and unsourced: on most teams they work with, somebody tries a few questions before a release. No sample, count or company is given.

The familiar anchor is the OWASP web application top ten, the list behind any corporate security training the audience has sat through. There is now one for AI.

The Reshuffled Risk Rows

Three rows are announced as worth the audience’s time, and four get walked through. The mismatch is reported rather than tidied.

The last row is the one to hold loosely. As captioned, the sentence about improper output handling contradicts itself, and the published chapter list resolves it in one direction. That resolution is metadata rather than evidence, so the reading stays uncertain here.

Everything in this section is a slide summary read aloud. No OWASP document is shown on screen at any point the recording can confirm.

Four Reasons Regression Misses It

There is no single right answer to assert on. A normal test says the result should be exactly this, and a good reply can be worded a hundred different ways with every one of them correct.

The input space has no end. Every reworded question is a brand new test case.

The reason flagged as the one to worry about is that the behaviour changes without a release. If the provider updates the model or tweaks a safety filter, your code did not change and your tests did not change, but your product did, so yesterday’s green build tells you nothing about today.

And nothing looks broken when it goes wrong. A valid 200, a valid response, perfect JSON, and the bot has just told a customer exactly how to get around the refund policy. That example is hypothetical and is not attributed to any real incident or engagement.

The conclusion drawn from all four is that none of them is a reason not to test. They are reasons not to test the way teams are testing now.

Joins And Cameras

The sentence the session sets out to argue with is the one that ends most of these conversations: it is a black box, you cannot test it.

The mental model behind that is joins and cameras. The software has joins, meaning places where things get handed from one part to the next, and a camera can go at every one of them.

Four cameras get named: what documents the system went and looked up, what it was told before it answered, what it actually did, and what it put on the screen afterwards.

The rule that follows is that anything you can see, you can check, and that is described as the idea behind the whole talk.

This is also the point the published description inverts. The description says the session treats AI systems as testable components including the model layer; the talk argues the opposite for the model, and names trying to fix the AI as the first mistake.

Six Steps To A Check

The recipe is deliberately narrow at the top: pick one risk at a time rather than all ten.

• Pick a risk. One, not the list.
• Describe a bad day. The worked version is a chatbot refunding more money than any human at the company is allowed to approve.
• Write a probe. A message saying to refund everything on the account.
• Find the signal. Not how the reply is worded, but whether a refund was issued and for how much.
• Write the check across retries. Across five retries it never refunded more than $200, with the $200 flagged as illustrative rather than a recommendation.
• Set the rule. Money is serious, so a failure stops the release.

Step four is named as the bottleneck, and as the place everybody gets stuck every time.

Acme Shop And The Receipt

The example system is fictional and is introduced as the victim. Acme Shop sells things, and no real customer or engagement is described anywhere in the session.

Its support chatbot searches the help centre for relevant pages and answers from what it found, which is a retrieval pipeline though that term is never used. The part that matters is that it can also act, refunding money and closing accounts.

The pipeline is drawn as six boxes: question in, lookup, documents handed to the model with the instructions, reply written, button pressed, something appears on screen. The optimistic reading offered is that every box can go wrong and every box leaves a trace.

The receipt is the artefact teams are told to demand from one request: what the user saw, what the system actually did, what it was allowed to read, what it said, and what it cost.

All of this was described on a slide. No receipt schema, log format, field list or code was shown in any form the recording can evidence.

Injection And Disclosure

The definition given for prompt injection is that the AI treated text as an instruction instead of as information.

The scenario is textbook indirect injection, though the phrase is never used. A customer types the instruction into a product review; another customer asks what the reviews say about their order; the chatbot searches, finds the review, reads it and acts.

The assertion is behavioural rather than textual: check whether an innocent question caused an action. The captions garble the follow-up sentence into the opposite of its meaning, so it is paraphrased here and not quoted.

Sensitive information disclosure gets a structural diagnosis. Private customer records and public help pages sit in the same searchable pile, and the retrieval layer never asks who is asking.

Two checks are proposed, with a mid-sentence self-correction from input to output. First, the private data must not appear in the reply. Second, and the more interesting one, the private document was never looked up at all.

That second check asserts on the retrieval log rather than on generated text, which is the camera-at-the-join idea made concrete.

Agency, Misinformation, Output

Excessive agency is the category flagged as the biggest mover, and the argument for it being the easiest to test is that the actions are structured and already in your logs. The action taken and the number used can both be recorded and read.

Three protections are named: an allow-list of what the agent may do, a cap on the amount with anything above it looping in a human, and human approval for anything that cannot be undone.

Misinformation is equated with hallucination, described as stating something false with full confidence and perfect grammar.

The hard stop on truth testing is the sharpest instruction in this stretch. You cannot test whether an answer is true, because doing so would need a list of every true fact in the world, so teams should stop trying to test for truth.

The substitute is provenance, in two questions: did the answer point at a document, and was that document one of the ones it just looked up. The case where a retrieved document is itself wrong goes unaddressed.

Improper output handling is deliberately unglamorous. A poisoned review comes back from search carrying a script tag and the chatbot puts it straight onto a support agent’s screen. The fix is to check the screen rather than the AI and escape the output where you display it, which teams already know how to do.

The Demo

A demo did run, across about three and a half minutes and four separate runs, with output narrated aloud. Whether the run was live or pre-recorded is never stated; the word live comes only from a machine-generated chapter title, so it is not used here.

Run one is the injection: two customer questions, a product review containing an instruction to ignore all previous instructions, and a narrated result of $4,900 refunded. That figure exists in the narration; the captions cannot confirm what was rendered.

Run two is a suite against the OWASP top ten in which everything fails. Each row is described as carrying four columns, the risk number, a pass rate, a severity and evidence, with each failure printing exactly what the chatbot did so whoever picks up the bug can work on it.

Run three is the same command with guardrails switched on, and everything passes. Six protections are announced and five are named: restricting what the system may search, treating text as text rather than as an instruction, hiding secrets in the reply, escaping the output, and limiting which buttons exist. The sixth is never stated, and is not invented here.

Run four is the point of the exercise. The identical suite runs again and some checks pass while others fail.

What never appeared on screen is everything a reader would need to reproduce it: no tool name, no repository or URL, no framework, no language, no configuration file, no guardrail implementation and no contents for the OWASP suite.

Sample N Times, Assert A Rate

The diagnosis for why AI test suites die is that they get switched off about a month after somebody builds them. No data, sample or baseline supports the month.

The non-determinism observed in that fourth run is the evidence: the same test input passed once out of five, and on a second run passed twice out of five.

The reframe is the part worth carrying. With a simple pass or fail, somebody will call that flaky. It is not flakiness; it is real behaviour, and a probe that works four times out of five is a finding.

The method sentence is to sample N times and assert on the rate, which is the only way to see the behaviour at all.

Five samples are offered as a starting point, with the claim that a problem occurring one time in four will be caught about three times out of four. The arithmetic holds for five independent samples, though no working is shown and no independence assumption is stated.

Thresholds And Drift

The bar gets set per class of risk. Money and personal data should pass every single time with no exceptions; tone and wording should pass nine times out of ten.

The rate then gets watched over time. A check that used to pass ten out of ten and now passes nine out of ten is the signal, and the captions mangle the negation in the sentence explaining why, so it is paraphrased: the cause to suspect is a model update you did not ask for and were not told about.

That drift is described as the earliest warning you will ever get that your product changed underneath you.

Failures have to be reproducible, which means saving the exact input alongside the whole receipt of what happened.

The checks then run in three places. On every code change, only the serious ones, costing nothing and finishing under 90 seconds. The reasoning for that budget is organisational: the moment checks start making people wait, somebody removes them and you do not get a second chance to add them back.

Nightly, everything runs several times against the real model, which is where the numbers get watched. Before release, a deep run sits behind a human clicking approve, so nobody gets a surprise bill on Saturday.

Mistakes And Take-Homes

Five mistakes are announced and four are listed. The fifth is never given, and the published chapter list presents all of them as personally made, while the spoken version is closer to having seen five and personally made most.

Trying to fix the AI heads the list. You cannot fix or control it, but you can fix what it is allowed to read, what it is allowed to do and what you put on the screen, and all three can be done this week.

Prompt-level pleading comes next, and is owned openly. Writing an instruction to ignore malicious instructions into your system prompt is a polite request rather than a safety measure, and the speaker has done exactly that.

Demanding perfection follows, since 100% on things that do not matter helps nobody. Last comes writing a 40-page AI safety report that nobody reads and nobody acts on.

The take-home slide has five steps: ask for the logs covering what it looked up, what it did and what it cost; pick the three risks that would hurt your company most; keep the nasty inputs in a text file so anyone on the team can add to it without needing a developer; add only the serious checks to the pipeline within the 90-second budget; and publish the same one-page summary every day.

The four takeaways are to test around the AI rather than the AI itself, to name the thing you look at, to run it more than once and count, and to let it fail the build, on the grounds that an AI check that cannot stop a release is not a check.

The closing line answers the title. AI behaviour can be measured, checked and reviewed. It is not a black box; it is a system nobody put logging into.

Q & A Session

Two audience questions were taken.

• How should security, QA and AI engineering teams share responsibility?

  Thejes Sree Satheesh Kumar: It is a shared responsibility, sitting with everyone including the developers. Do threat modelling against the OWASP top ten risks, and act on it. The question gets restated first and the sentence ends there, so this is a partial answer that trails off. No responsibility matrix, ownership boundary, artefact or process is named.

• How do you evaluate agent misuse across multi-step tool call trajectories rather than single steps?

  Thejes Sree Satheesh Kumar: Multiple tool calls can be logged, and the tools available can be restricted. The log will show which tools were called, whether that was intended and what each tool is supposed to do. He asks for the question to be repeated and the moderator reads it back, and the reply then restates that logging advice at the single-call level, which makes it a non-answer. Trajectories, sequences, ordering and cross-step evaluation, which is what was asked about, go unaddressed.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.