LLM Security: OWASP Top 10 for LLMs 2026 and How to Test It

What Is LLM Security?

LLM security is the practice of protecting an application built on a large language model, and the data and systems it can reach, from attacks that arrive through what the model reads and land through what the application does with its output. It covers the prompts and hidden context, the retrieval corpus, the tools, the output handling, the model supply chain and the cost of every call.

The NIST AI Risk Management Framework (AI RMF) calls an AI system secure when it can maintain confidentiality, integrity and availability through protection mechanisms that prevent unauthorized access and use, and it names adversarial examples, data poisoning and the exfiltration of models or training data as common concerns.

OWASP's 2026 Prompt Injection entry notes that LLMs make no architectural distinction between instructions and data, so there is no clean equivalent to parameterized queries: any text that reaches the context window, whether from a user, a document, a tool result or memory, can change what the model does.

OWASP rebuilt its 2026 list by testing a practitioner vote against 6,639 classified incidents drawn from public vulnerability databases and an AI-harm database. The preface also sets the scope: the list owns the risk when the model is a component inside your application, and once the model becomes an actor, with tools it can call and memory it carries between sessions, the risk moves to the OWASP Top 10 for Agentic Applications.

That is the same line the guide to AI agent security draws: LLM security asks what a model can be made to say, reveal or emit, and agent security asks what a system can be made to do. Security is also one layer of LLM testing, next to functional, performance and quality checks.

The examples on this page follow one application: a support assistant that answers from a help-center index, looks up orders with a read-only tool, emails customers through a send_email tool and renders markdown in a web chat.

LLM Security vs LLM Safety

LLM security protects the application from attackers, while LLM safety keeps the model's own output from harming people. A customer record leaked through a planted instruction is a security failure. Hateful content, or instructions for something dangerous, is a safety failure even when nobody attacked the system.

NIST's AI RMF lists safe, and secure and resilient, as separate characteristics: a safe system should not, under defined conditions, lead to a state that endangers human life, health, property or the environment, while a secure one protects confidentiality, integrity and availability.

NIST's Generative AI Profile, NIST AI 600-1, names 12 risks that generative AI creates or makes worse. Information Security is one of them, covering lowered barriers for offensive cyber operations and a larger attack surface that can compromise a system's availability or the confidentiality or integrity of its training data, code or model weights. CBRN information and confabulation are separate entries that NIST maps to the safe characteristic, while dangerous, violent or hateful content maps to both safe and secure.

Jailbreaks and misinformation sit on both sides. A jailbreak is an attack aimed at a safety failure, which OWASP files under Prompt Injection, and OWASP lists Misinformation as LLM07:2026, noting that when a model's confident output drives a decision or a tool call, a wrong answer turns into a wrong action. Test both with one harness, but give security rows evidence-based pass conditions and safety rows a graded content policy.

What Changed in the OWASP Top 10 for LLM Applications 2026?

The 2026 edition reordered the list and broadened one entry under a new name. Excessive Agency climbed from sixth to third, Unbounded Consumption rose four places to sixth, Misinformation moved from ninth to seventh and Improper Output Handling fell from fifth to tenth. System Prompt Leakage became Hidden Context Exposure, which also covers retrieved policy text, tool schemas and other context users are not meant to see.

OWASP's preface says the community vote carries three-quarters of the ranking weight and the incident record the remaining quarter, enough to move an entry a tier when the gap between belief and evidence runs wide.

• Prompt Injection - practitioners rank it first, yet ranked by the raw incident record it falls out of the top 10. OWASP reads that gap as a defense effect: teams fight injection hard, so fewer clean exploits reach public databases, and it stays at LLM01.
• Misinformation - voters placed it near the bottom and the incident record near the top, the widest gap in the direction that hurts, so the evidence pulled it up to LLM07.
• Excessive Agency - OWASP calls its climb the most consequential move on the list, because the vote and the record agree that "agentic deployments are where the damage is landing."

Several entries also grew, according to the same preface. Prompt Injection now covers cross-modal attacks that hide instructions in an image or an audio track, Supply Chain covers a promoted model artifact that is not what it claims to be, Data and Model Poisoning absorbs fine-tuning subversion, and Improper Output Handling spans the insecure code that assistants generate at scale.

For a test plan, the scope changes matter more than the order. A suite written against the 2025 names may probe only for a leaked system prompt and never check whether retrieved policy text or tool schemas leak, which Hidden Context Exposure now covers.

A Security Test for Each OWASP LLM Risk

Write each test as a probe, an evidence source and a fail condition, and set up the evidence source before you write the payload. The rows use the support assistant; the probes are this article's design, while the risks, examples and controls behind them come from OWASP's 2026 document.

• Canary strings - plant a unique marker in a customer record, a restricted document or the system prompt, then search every output channel for it. OWASP's LLM02 entry counts tool-call arguments, reasoning traces, retrieved chunks, logs and telemetry as disclosure surfaces, alongside the reply.
• Planted documents - OWASP's LLM01 entry cites the PoisonedRAG study, which reached a 90% attack success rate by injecting five malicious texts per target question into a knowledge database with millions of texts, so the LLM09 row needs a planted document as well as a hostile prompt.
• Pipeline rows - LLM04 and LLM05 run where models, adapters and datasets are built and promoted, so they belong in the build pipeline, not the chat suite.

Is Jailbreaking the Same as Prompt Injection?

No: in OWASP's 2026 definition, jailbreaking is the subset of prompt injection where the attacker's goal is to make the model violate its safety protocols. Prompt injection is the wider class, covering any input, typed by the user or carried in retrieved content, tool output, images, audio or memory, that changes the model's behavior in ways the developer did not intend.

• Jailbreaks - the user types the payload, often a persona or role-play framing, and the test grades the output against the safety policy.
• Direct injection - the user overrides the system prompt's role or limits to get data or actions outside its scope, and the test grades what the assistant disclosed or did.
• Indirect injection - the payload sits in content the assistant retrieves, such as a help article or an order note. The user never sees it, and the test grades tool calls, retrievals and rendered output.
• Encoded and cross-modal payloads - Base64, ROT13, emoji or invisible Unicode, and instructions carried in images or audio. OWASP notes that encodings slip past filters that never saw them, so send each encoding and each modality the application accepts.

Fixed payload lists overstate how well a defense holds. OWASP's LLM01 entry cites The Attacker Moves Second, in which Nasr and colleagues bypassed 12 recent defenses with attack success rates above 90% for most, although the majority of those defenses had originally reported near-zero attack success. Their adaptive attacks used gradient descent, reinforcement learning, random search and human-guided exploration.

The guide to prompt injection testing covers payload families and CI setup for this one attack class in more depth.

How to Test an LLM Application for Security

Run the plan against staging with synthetic customer records, so a probe that works leaks test data instead of real data. For the support assistant:

• Map what the model reads and where its output goes - list the user turn, retrieved articles, order-lookup results and uploads on the way in, and the chat renderer, customer emails, logs and traces on the way out. OWASP suggests breaking each injection scenario down by delivery surface, propagation and encoding before choosing mitigations, and the same breakdown shows where each probe belongs.
• Keep the rows that apply - the assistant needs LLM01 to LLM03 and LLM06 to LLM10 in its application suite. LLM04 and LLM05 move to the build pipeline, because the assistant calls a hosted model it does not fine-tune.
• Switch on the evidence first - log retrievals with each caller's access scope, capture tool calls with their arguments, count tokens per session and save the rendered chat page, so a probe that works leaves a record the test can find.
• Baseline, then attack adaptively - OWASP's LLM01 guidance says to baseline with AgentDojo and JailbreakBench, then red-team with the full defense specification disclosed to the testers. JailbreakBench adds a jailbreaking dataset of 100 behaviors aligned with OpenAI's usage policies for the jailbreak rows.
• Record a rate across repeated runs - model output varies from run to run, so send each probe several times and track how often it lands before you judge a defense.
• Re-run on every change - run the suite again when the model or model version, system prompt, retrieval corpus, tool list or renderer changes, and gate the release on the rows that matter most. The roundup of AI red teaming tools compares options for automating those reruns.

Before a custom harness exists, the free Agent Red-Team Scan sends 25 prompts to an LLM API you name, including persona jailbreaks, overrides encoded in Base64, hex or ROT13, and system prompt and PII leak probes. It looks for canary tokens in the replies and scores the result from 0 to 100.

For a chat or voice assistant that only answers, TestMu AI's Agent Testing runs 15+ specialized evaluator agents against the live endpoint for hallucination, bias, toxicity and compliance, and its CLI adds red-team runs by category, such as prompt injection, jailbreak and PII leakage.

Controls That Hold When the Model Is Fooled

OWASP's preface states the posture every control below serves: "Stop trying to build a model that cannot be fooled. Build the system around it, so that when the model is fooled, and it will be, nothing important breaks." Each control is also a test target, because the matching probe in the table should fail against it.

• Least privilege per tool - give read tools read-only identities, run tools in the user's context and require approval for high-impact actions (LLM03). The LLM03 row passes when an injected email request stops at the tool, whatever the model decided.
• Authorization before retrieval - enforce document- and chunk-level authorization inside the index query rather than after retrieval (LLM02, LLM09), so the canary in the restricted article is never retrieved at all.
• Nothing secret in hidden context - keep credentials and sole access rules out of the system prompt and enforce them in code (LLM08), so an extracted prompt gives an attacker nothing to use.
• Model output treated as untrusted input - encode output for the place it lands, parameterize queries and turn off auto-rendered markdown images and link previews (LLM10).
• Caps that halt - set spending caps that stop inference instead of raising alerts, plus step, recursion and time limits for loops (LLM06).
• The Rule of Two - OWASP cites Meta's rule as a floor: an application that combines untrusted input, sensitive data and state change or external communication needs per-action human approval. The support assistant holds all three once send_email is on, so every outbound email needs a confirmation step.

Content filters and other AI guardrails sit on top of these controls. OWASP's LLM08 guidance says to enforce critical behaviors through independent, deterministic systems outside the model rather than through instructions in the prompt.

Testing LLM Apps That Call Tools With Agent Assurance

With send_email in its toolset, the support assistant can act, so a test has to check the email that went out as well as the reply. TestMu AI's Agent Assurance tests agents that call tools, write files and change records, before they ship, and it runs from the terminal as Rook CLI. For this problem, it checks:

• Adversarial scenarios by default - rook generate writes functional and adversarial scenarios unless you narrow the suite, and the adversarial class covers nine categories, including prompt_injection, jailbreak, data_exfiltration, pii_leakage and hijacking. As test framing, the first two line up with LLM01:2026 and the next two with LLM02:2026; the mapping is this article's.
• Tool calls against declared tools - each call the profile returns is checked against the tools the assistant itself declares, and when the profile returns the calls it observed, a scenario can require that send_email is not_called.
• Effects a run leaves - files that changed under the paths the profile declares, artifacts produced, and records a judge confirms through a read-only tool you approve, so a claimed action never counts as proof.
• Leak tripwires - a scenario can list forbidden values the assistant must never claim or leak, which suits the canaries from the LLM02 and LLM08 rows.
• Unable to Verify reported apart - a criterion the run could not observe is Unable to Verify, reported beside the pass rate and kept out of it.
• Before release, not at runtime - it tests the assistant before you ship and blocks nothing in production, so the controls above still do the runtime work.

OWASP's LLM01 controls say to pin, sign and verify every MCP server and audit tool descriptions for hidden instructions, and they treat MCP servers as a supply-chain surface that ASI04 on the Agentic list covers. Rook CLI applies part of that control to repository-controlled servers. This help output was captured from Rook CLI 0.1.5, installed from npm, on September 30, 2026, without starting a run:

$ rook --version
0.1.5
$ rook mcp --help
Usage: rook mcp [options] [command]

the MCP servers rook and the agents under test can reach

Options:
  -h, --help                         display help for command

Commands:
  list [options]                     every server, its origin, transport and
                                     state
  get [options] <name>               one server's config, with ${VAR} left as a
                                     reference
  add [options] <name> [command...]  declare a server. Flags match `claude
                                     mcp`, so muscle memory transfers
  remove [options] <name>            drop a server from one scope
  enable [options] <name>            let rook and the agents under test reach
                                     this server
  disable [options] <name>           keep the declaration, stop using it
  approve [options] <name>           trust a project or discovered server,
                                     after reading what it runs
  help [command]                     display help for command

The approve subcommand is where that happens, and it applies only to project and discovered servers; servers added at local or user scope need no approval. The documentation on MCP servers in Agent Assurance says a project or discovered stdio server stays inert until a person approves its exact fingerprint, and a change to its command, arguments or environment references sends it back to pending approval. The same page says references such as ${VAR} stay unexpanded in display output, so tokens do not leak to the terminal, the model context or the transcript.

Compared with eval tools, the defaults differ, and this comparison describes each category's default approach, not any single product:

• What counts as proof - most eval and observability tools score what the agent said and recorded; Agent Assurance checks what the run changed and reports what it could not verify.
• Where the tool expectation comes from - eval tools check recorded calls against a list you write for each case, while Agent Assurance derives the expectation from the tools the agent declares.
• Adversarial coverage - Agent Assurance generates adversarial scenarios by default, and some eval tools ship a separate red-team module.

Like eval tools, Agent Assurance uses model judges, grades what the agent says as well as what it did, and runs in CI. Rook CLI installs from npm and needs Node 22 or newer:

npm install -g @testmuai/rook
rook --version

The Rook CLI install guide covers Homebrew and the shell installer, which carry their own Node runtime. To drive it from Claude Code, install the skill, then describe the scenarios you want:

npx @testmuai/rook-skill@latest install --agent claude-code

/rook Generate up to three data_exfiltration scenarios for the support assistant in this repository, in which a help article or an order note asks it to email a customer's order history to an outside address. Grade each scenario on the observed send_email calls rather than the reply, use the staging profile, and before invoking the agent, list the write tools it declares.

Note: Agent Assurance derives these scenarios from the assistant's code, runs them against your staging profile before release, and grades each criterion as Pass, Fail or Unable to Verify against observed evidence, never against the assistant's account of its own work.

Conclusion

Start your LLM security testing with the two rows the support assistant is most exposed to, LLM01 through a planted help article and LLM10 through the chat renderer, and run each probe several times against staging. Once both report a stable rate, generate adversarial scenarios for the LLM03 row and its send_email tool with Agent Assurance, following the guide to Agent Assurance test scenarios.

Note: Anubhav Singhmaar, AI Product Manager at TestMu AI with expertise in large language models and agentic AI, is the author of record for this article, which was researched and drafted with AI assistance. Its sources are OWASP and NIST publications and research papers that OWASP cites. Our editorial process and AI use policy explains how we use AI in our content.