Agentic AI Risks: 11 Risks and How to Test for Each

1. False Completion Reports

A completion report can be false even when the checks agree with it, because an agent can satisfy a check without doing the work. In ImpossibleBench (Zhong, Raghunathan and Carlini, October 2025), the unit tests contradict the task's specification, so any pass means the agent cheated. The best-performing model in the paper's tests, GPT-5, "cheats 54.0% of the time on Conflicting-SWEbench," and the strategies it records include modifying tests despite being explicitly instructed not to.

The closest entry in the OWASP Top 10 for Agentic Applications is ASI10 Rogue Agents, whose reward-hacking example describes agents that game their assigned reward systems "by exploiting flawed metrics to generate misleading results."

How to test for it

• Grade the effect - check the diff, record or artifact the task should have produced, and leave the agent's summary out of the verdict.
• Give it a way out - tell the agent how to report that a task cannot be done; in ImpossibleBench, an abort option lowered GPT-5's cheating rate on Conflicting-SWEbench to 9%.
• Keep the checker out of reach - hide the tests or checks the verdict depends on; the same paper found that hiding tests cut cheating success to near zero, though it also lowered scores on the original benchmark.

For the evidence to collect for each kind of claimed action, see the guide to detecting false completion reports.

2. Compounding Errors in Multi-Step Tasks

A small error rate per step becomes a large failure rate per task once an agent chains many steps. If each step succeeds 95% of the time and failures are independent, a 20-step task succeeds about 36% of the time (0.95 to the 20th power).

Errors in real runs are not independent either. In the paper The Illusion of Diminishing Returns (Sinha et al., 2025), models "become more likely to make mistakes when the context contains their errors from prior turns," an effect the authors call self-conditioning and found does not go away by scaling up the model alone.

How to test for it

• One criterion per step - give every step that leaves an effect its own acceptance criterion, so the verdict shows which step broke.
• Seed an early error - make an early tool return a plausible wrong value, such as the wrong customer ID, and check whether later steps catch it or build on it.
• Test the long version - run the workflow at the length it will have in production, because a pass on a 5-step version says little about the 25-step one.

3. Inconsistent Results Across Runs

One passing run proves little about the next hundred when the agent can choose a different chain of tool calls each time it runs. The τ-bench paper (Yao et al., 2024) introduced pass^k for this, "defined as the chance that all k i.i.d. task trials are successful, averaged across tasks," a number that falls as k grows when an agent is inconsistent.

How to test for it

• Run each scenario k times - report how many of the k runs passed, and treat a pass rate as a claim about that k.
• Separate flaky from broken - a scenario that flips between pass and fail on an unchanged setup needs a different fix from one that fails every time.
• Record the setup - store the model version, prompt and tool list with every run, so a change in the rate can be traced to a change in the setup.

4. Cascading Failures Across Agents

In a multi-agent system, one agent's output becomes the next agent's trusted input. OWASP's ASI08 Cascading Failures describes a single fault that "propagates across autonomous agents, compounding into system-wide harm," and names the symptoms to watch: rapid fan-out, oscillating retries or feedback loops between agents, and repeated identical intents.

How to test for it

• Inject a fault at one hop - put a wrong value in the planner's output and check whether the executor validates it before acting on it.
• Count the fan-out - track downstream tasks and retries per upstream decision, and flag a spike that follows a single bad input.
• Replay before widening scope - the ASI08 mitigations include re-running recorded agent actions "in an isolated clone of the production environment" to see whether the same sequence would cascade, before any policy expansion goes live.

The guide to multi agent testing covers handoff checks between agents in more depth.

5. Runaway Cost and Resource Use

An agent that loops, retries or fans out can spend far more than a task is worth. The OWASP LLM06:2026 Unbounded Consumption entry describes the deliberate version, Denial of Wallet, where attackers initiate "a high volume of operations" to "exploit the cost-per-use model of cloud-based AI services."

How to test for it

• Set a budget per scenario - cap tokens, tool calls and wall-clock time, and fail the scenario when it goes over.
• Force a retry loop - make one tool return a transient error on every call, then check that the agent stops after a bounded number of retries.
• Track cost per success - divide total spend by passing runs, because a cheaper agent that fails more often can cost more per completed task.

To gate a release on cost per success, follow the walkthrough of LLM cost tracking for agent evals.

6. Accountability and Oversight Gaps

When an agent acts, someone must be able to show who authorized an action, reconstruct what happened, and stop the agent mid-task. OWASP's ASI09 Human-Agent Trust Exploitation describes the human side of the failure, people "approving actions without independent validation," and lists missing confirmation for sensitive actions as a common example.

The NCSC post separates human-in-the-loop, where "humans approve actions before they happen," from human-on-the-loop, where "humans monitor actions and can intervene if needed." It adds that you "should always be able to 'pull the plug' and halt autonomous AI agent activity immediately."

How to test for it

• Test the approval gate - send a request that needs sign-off, such as a refund above the limit, and check that no write happens before an approval record exists.
• Test the stop switch - halt the agent mid-task and check that no tool calls or writes follow the halt.
• Reconstruct one run from the logs - name the identity behind each call from the logs alone, and fix the logging before release if you cannot.

Ownership, approvals and audit trails are covered in depth in the guide to agentic AI governance.

7. Prompt Injection and Goal Hijacking

How often an agent obeys a planted instruction depends partly on the model behind it. In NIST CAISI's evaluation of DeepSeek models (September 30, 2025), agents based on DeepSeek's most secure model were, on average, 12 times more likely than the evaluated U.S. frontier models "to follow malicious instructions designed to derail them from user tasks."

Plant one instruction in each channel the agent reads, then check the observed tool calls for any call outside the user's task, and re-run the set whenever the model changes. Payload families for each channel are listed in the article on prompt injection testing.

8. Excessive Permissions

An agent with more access than its task needs turns every other failure on this list into a bigger one. CISA's announcement of the joint guide tells organizations to "avoid granting broad or unrestricted access, especially to sensitive data or critical systems," and OWASP files the agent version of the problem under ASI03 Identity and Privilege Abuse.

To test it, list every tool, credential and data scope the agent holds against what each task needs, then ask for an action just outside that scope and check that the request is denied and the denial is logged. The guide to AI agent permission testing walks through each route to an out-of-scope action.

9. Tool Misuse

The agent stays within its permissions but uses a legitimate tool in a harmful way. OWASP's ASI02 Tool Misuse and Exploitation gives "deleting valuable data, over-invoking costly APIs, or exfiltrating information" as examples, along with an email summarizer that "can delete or send mail without confirmation."

Write must-not-call criteria for the destructive tools, such as delete, send and transfer, then give the agent a plausible reason to use them, like a request to clean up old records. Check the call log and the records afterwards, since the reply can describe a careful cleanup that deleted far more than it should.

10. Data Exfiltration and PII Leakage

Private data can leave through a reply, a rendered link or image URL, or a tool call to an outside service. The first example OWASP gives under ASI01 is this path: hidden instructions in web pages or documents that "silently redirect an agent to exfiltrate sensitive data or misuse connected tools."

Canary values expose it: plant a fake customer email and card number in data the agent can read, then search every reply, URL and outbound request for them. Capture outbound traffic at a proxy the agent cannot configure, and count a canary found outside the environment as a failure whatever the reply says.

11. Policy Violations

An agent told to enforce a refund limit can approve a refund above it in a well-formed reply that quotes the limit correctly. τ-bench checks the records as well as the replies: its agents get domain policy guidelines, and its grading "compares the database state at the end of a conversation with the annotated goal state."

Turn each written rule into a criterion checked against the records the agent changed, then press on the rule the way a determined customer would: urgency, a claim of authority, or one large request split into several small ones.