AI Hallucination: Causes, Examples and How to Reduce It

What Is an AI Hallucination?

An AI hallucination is output from an AI model that is false, or not supported by anything the model was given, yet is presented as fact in the same fluent, confident style as a correct answer. The term covers text, citations, code, speech transcripts, images and, for AI agents, reports of actions that never happened.

The confidence is what separates a hallucination from a failure you can see. The Tow Center for Digital Journalism gave eight AI search tools excerpts from news articles and asked each to name the article, its publisher, its date and its URL. Its report in Columbia Journalism Review (March 2025) says the tools "provided incorrect answers to more than 60 percent of queries", and that ChatGPT Search misidentified 134 of 200 articles, signaled a lack of confidence fifteen times and never declined to answer.

A hallucinated output always contradicts something, and naming that something is the first step toward catching it:

• The world - the output states something untrue about reality: a wrong date, a court case that was never decided, a study that was never published.
• A source it was given - the output adds to or contradicts a document, search result or database row that was in front of the model. The statement may be true somewhere else and is still unsupported here.
• Its instructions or its own earlier words - the output ignores what you asked for, or contradicts something the model said earlier in the same conversation.

A statement can also be wrong without being a hallucination, for instance when the source the model was given is itself wrong and the model reports it accurately. The guide to LLM hallucination detection separates a hallucination from a factual error and compares the methods that flag each.

Types of AI Hallucination, With an Example of Each

The most useful way to sort AI hallucinations is by what the output contradicts, because that names the reference you compare it against. The table below builds on two published taxonomies. A survey by Huang and co-authors in ACM Transactions on Information Systems splits hallucination into factuality (conflict with real-world facts) and faithfulness (divergence from the user's input, or from the output itself), with instruction, context and logical inconsistency as the faithfulness subtypes.

A content analysis of ChatGPT errors in Humanities and Social Sciences Communications (September 2024) went finer: it identified 8 first-level error types, subdivided into 31 second-level error types. The table uses a coarser set, chosen for testing.

Read the last column before you choose a test. It tells you what you have to collect first: questions with known answers, the documents the system retrieves, the prompt, the conversation history, or access to the system an agent writes to. If you cannot name the reference for a type, you cannot yet measure that type.

Generated code has its own version of the first two rows: packages and API methods that do not exist. The guide to AI code hallucinations covers how to catch those.

Causes of AI Hallucination, and the Fix That Targets Each One

AI models hallucinate for several separate reasons, and a fix aimed at one cause does little for the others. Retrieval, for example, supplies a missing fact and does nothing about a model that ignores the passage it was handed.

Rows three and four come from 2025 research that changed the standard account, which used to stop at flawed training data. Why Language Models Hallucinate (Kalai, Nachum, Vempala and Zhang, of OpenAI and Georgia Tech, September 2025) argues that models hallucinate "because the training and evaluation procedures reward guessing over acknowledging uncertainty". Under a scheme that awards 1 point for a correct answer and none for "I don't know", a model that always guesses outscores one that admits doubt.

The same paper puts a floor under the rare-facts row: "if 20% of birthday facts appear exactly once in the pretraining data, then one expects base models to hallucinate on at least 20% of birthday facts."

Anthropic's interpretability research, Tracing the thoughts of a large language model (March 2025), looked inside a Claude model and found that "refusal to answer is the default behavior": a circuit that says the model lacks the information stays on unless a "known entities" feature switches it off. A hallucination can follow when the model "recognizes a name but doesn't know anything else about that person". By switching on the "known answer" features for a name the model did not know, the researchers write, "we're able to cause the model to hallucinate (quite consistently!) that Michael Batkin plays chess."

Both findings change what you test. Put half-known names and unanswerable questions in your test set, and score a declined answer differently from a wrong answer, because a test that counts correct answers alone rewards the guessing you want to remove.

How Often AI Hallucinates, and What Each Published Rate Measures

No single AI hallucination rate exists. Every published figure is a count of one thing divided by a count of another, on one task, judged by one grader. The figures below are all real, and no two of them measure the same thing.

The figures disagree for reasons you can check in any report:

• Different tasks - answering from memory, citing a news article and researching case law are different jobs. Vectara's hallucination leaderboard measures a fourth: summarizing a document the model was handed.
• Different handling of "I don't know" - a declined answer can count as wrong, count as neutral or drop out of the denominator altogether.
• Different graders - a match against a reference answer, a detector model and a panel of journalists will not agree on what counts as a hallucination.

The guide to LLM benchmarks covers what public leaderboards measure in general. To show the second reason on its own, a short script was run on 7 October 2026 with Node.js v25.5.0. Its input is sample data: two imaginary systems, the same 20 questions, and a grade for each answer (correct, wrong or abstained) written by hand.

node rates.mjs

20 questions, two sample systems
  System A (answers every question)
  System B (abstains when unsure)

                                         System A System B
----------------------------------------------------------
Correct answers                                13       12
Wrong answers                                   7        2
Abstained (said it did not know)                0        6

Accuracy: correct / all questions           65.0%    60.0%
Wrong answers / all questions               35.0%    10.0%
Wrong answers / questions it answered       35.0%    14.3%
Abstentions / all questions                  0.0%    30.0%
Score if a wrong answer costs a point           6       10

Ranked by accuracy alone:              System A first
Ranked by fewest wrong answers:        System B first
Ranked with wrong answers penalized:   System B first

The sample set measures nothing about any real model, and 20 questions is far too few for a real comparison. It shows how the counting rule changes the headline number:

• System A has the better accuracy (65.0% against 60.0%) and gives 7 wrong answers where System B gives 2.
• For System B, wrong answers over all questions is 10.0% and wrong answers over the questions it answered is 14.3%. Both are honest rates from the same 20 grades.
• Ranked by accuracy alone, the system that guesses comes first. Under either rule that counts or penalizes wrong answers, the system that abstains comes first.

The first row of the table is the same effect in published data. By OpenAI's figures, o4-mini had the slightly higher accuracy (24% against 22%) and nearly three times the error rate (75% against 26%), because it abstained on 1% of questions where gpt-5-thinking-mini abstained on 52%. The post draws the conclusion itself: "Strategically guessing when uncertain improves accuracy but increases errors and hallucinations."

Before you compare two published rates, find the numerator, the denominator and what happened to the declined answers.

Real AI Hallucination Examples and Their Consequences

Fabricated court citations, as in the case that opens this guide, are documented decision by decision in Charlotin's database. The examples below come from other domains. Each is taken from a primary document or the first report, with what that source does and does not establish.

Speech-to-text in hospitals (2024) - the paper Careless Whisper, presented at ACM FAccT 2024, tested OpenAI's Whisper on recordings from TalkBank's AphasiaBank in April and May 2023. It found that "roughly 1% of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio", and that 38% of those hallucinations included explicit harms such as perpetuating violence or implying false authority.

An Associated Press investigation published on 26 October 2024 reported that more than 30,000 clinicians and 40 health systems had started using a Whisper-based tool to transcribe patient visits, and that the tool erased the original audio. The AP report describes no fine or ruling, and it noted OpenAI's own warning against using Whisper in "high-risk domains".

A consulting report for a government (2025) - The Guardian reported on 6 October 2025 that Deloitte would give the Australian government a partial refund "over a $440,000 report that contained several errors, after admitting it used generative artificial intelligence to help produce it". That figure is the value of the report in Australian dollars, and the refund amount had not been made public when the story ran.

The errors, reported by the Australian Financial Review in August 2025, included references to academic reports that do not exist and a made-up reference to a court decision. Deloitte "did not state that artificial intelligence was the reason behind the errors"; the University of Sydney academic quoted by The Guardian, Dr Christopher Rudge, described them as hallucinations.

A product launch (2023) - in Google's own promotional material for its Bard chatbot, Bard said the James Webb Space Telescope "took the very first pictures of a planet outside of our own solar system". The Verge reported on 8 February 2023 that the first image of an exoplanet was taken in 2004.

The Guardian reported the next day that Alphabet stock "slid by 9% during regular trading in the US". The fall followed the error and, by the same report, came amid investor fears about Microsoft's competing search product. Google's statement to The Verge named the remedy: "a rigorous testing process" so that responses "meet a high bar for quality, safety and groundedness in real-world information."

A false accusation against a real person (2025) - on 20 March 2025 the privacy group noyb filed a complaint against OpenAI with Norway's data protection authority. Asked about a Norwegian man, ChatGPT had presented him as a convicted criminal who murdered two of his children, mixed with real details such as the number of his children and the name of his home town. That is a complaint, not a ruling, and noyb's page reported no decision on it as of 7 October 2026.

In the United States, a Georgia court granted summary judgment to OpenAI on 19 May 2025 in Walters v. OpenAI, a defamation claim over a ChatGPT output that named a radio host as an embezzler. The order records that Walters "testified that he incurred no damages".

In the first three examples a reference could have exposed the error: the recording, the cited works, a NASA page. The check was skipped, or in the transcription tool's case the recording had been erased.

How to Test an AI System for Hallucinations

A hallucination test compares an answer with a reference and records one of three outcomes. Its parts are the same whatever the system:

• Input - a question with a known answer, or a question together with the source the system must answer from.
• Output - the system's answer, captured exactly, and captured on several runs because the same question can produce different answers.
• Reference check - the answer compared with the known answer or, claim by claim, with the source.
• Outcome - correct, wrong or declined.

Score "declined" as its own outcome. If a declined answer counts as wrong, the test pushes you toward a system that always answers, and the sample output above shows what that hides: the system that guessed looked better on accuracy and gave more than three times as many wrong answers. If declined answers are left out altogether, a system can look safe by answering almost nothing.

Published benchmarks that separate the outcomes work the same way. In SimpleQA, the question set behind the first row of the rates table, each answer "is graded as either correct, incorrect, or not attempted." Groundedness scoring and judge models automate the reference check at scale, and each fails in its own way, which the LLM hallucination detection guide compares.

For the step-by-step version on a deployed support bot, from building the question set out of your own knowledge base to deciding which results block a release, see the guide to chatbot hallucination, which also splits "declined" into a correct refusal and an over-refusal. Hallucination tests are one part of a wider LLM testing plan, next to regression, safety and performance tests.

Hallucinated Actions: Checking What an AI Agent Did With Agent Assurance

Most of the types above are text you can check against other text: a reference, a supplied document, the prompt or the conversation. An agent with tools can also hallucinate an action: its final message says a record was updated or a message sent, and the step never ran. A 2025 survey of hallucinations in LLM-based agents defines execution hallucinations as cases where agents "claim to have completed certain sub-stages during the execution phase, but in reality, they have not actually been performed or accomplished", one of five types of agent hallucinations it classifies.

Take an agent that works an engineering issue tracker and is asked to close a resolved ticket. It replies that the ticket is closed, and the ticket is still open because the close call returned an error the agent did not act on. The reply, the transcript and a judge model's score of either are all the agent's own account, so the reference has to be the observed tool calls and the ticket system itself; the guide to agent action hallucination covers the failure modes in depth.

TestMu AI's Agent Assurance tests agents that act, before release, with the agent running against staging. It grades each acceptance criterion against evidence, and for a claimed action that evidence is:

• Tool calls - the calls the profile returns from a run, checked against the tools the agent declares, including calls that must never happen.
• Files and artifacts - files that changed on disk under the paths the profile declares, and the artifacts the run produced.
• Records - when the agent says it wrote a record, a judge can confirm the record with a read-only query through a tool you provide and approve.
• Hallucination scenarios - hallucination is one of nine adversarial scenario categories. The adversarial class is generated by default, and you can narrow a suite to the classes or categories you want.
• Verdicts - each criterion is Pass, Fail or Unable to Verify. Unable to Verify is not a failure and stays out of the pass rate.

How much Agent Assurance can observe depends on the profile and the access you give it. A profile that returns only the agent's answer leaves most action criteria Unable to Verify.

Most eval and observability tools score what your agent said and recorded. Agent Assurance checks what the run changed, and reports what it could not verify. That contrast describes each category's default approach, not any single product, and an eval tool remains the right check for an invented statement in an answer.

Agent Assurance also uses model judges and grades what the agent says; a claimed action never counts as proof. It runs before release, so it is not a guardrail or a hallucination detector for live traffic.

Agent Assurance is pre-alpha and publicly installable. You run it from a terminal, where it is called Rook CLI. Install it from npm:

npm install -g @testmuai/rook

Rook CLI runs on macOS and Linux, and 64-bit Windows through npm or WSL. The npm route is the one that needs Node.js 22 or newer. Claude Code users can add the rook skill with one command:

npx @testmuai/rook-skill@latest install --agent claude-code

The Agent Assurance overview in the docs explains what counts as evidence and how verdicts are assigned. Point every run at staging, because the writes your agent makes during a test are real.

How to Reduce AI Hallucinations, and Whether They Can Be Eliminated

Each method below targets a cause from the causes table, and the published measurements show how large an improvement to expect.

• Ground the answer in a source, then check the answer against it - retrieval puts the missing fact in the prompt, and a claim-by-claim check catches the model ignoring it. RAG evaluation metrics such as faithfulness measure that second step.
• Let the system decline, and stop penalizing it for declining - tell it to say so when the source does not cover the question, and score a declined answer as its own outcome.
• Verify claims before they are shown - in Chain-of-Verification (Dhuliawala and co-authors, arXiv 2309.11495), the model drafts an answer, plans verification questions, answers them independently and then revises. On a Wikidata list task this more than doubled precision over the Llama 65B few-shot baseline, "from 0.17 to 0.36". That is a 2023 model, and at 0.36 most listed items were still wrong.
• Narrow the scope - a system limited to one product's documentation has fewer half-known names to guess about than a general assistant. Scope and output rules are one kind of AI guardrails.
• Keep a person in the loop where a wrong answer is costly - legal filings and published reports both appear earlier in this guide, and each passed through someone who did not check; in the transcription case the recording a reviewer would have needed had been erased.
• Test on every change - a new model version, a prompt edit or a knowledge-base update can move the rate, so rerun the same question set and compare.

Prompt instructions and a low temperature do less than expected. In a study in Communications Medicine (August 2025), six language models read 300 physician-validated clinical vignettes that each contained one fabricated detail, and a prompt written to reduce hallucinations lowered the mean rate from 66% to 44%, a real gain that still left the models elaborating on the invented detail in 44% of outputs. On temperature, the study reports: "Temperature adjustments offer no significant improvement."

Whether AI hallucination can be eliminated depends on the definition. Xu, Jain and Kankanhalli (arXiv 2401.11817, January 2024) define hallucination formally as any inconsistency between a model and a ground-truth function, and show that under that definition "it is impossible to eliminate hallucination in LLMs" used as general problem solvers. OpenAI's 2025 post agrees that accuracy "will never reach 100%" because some questions are unanswerable, and still rejects the claim that hallucinations are inevitable: "They are not, because language models can abstain when uncertain."

The two positions fit together. No model can be right about everything, and a system can be built and graded so that it says it does not know instead of inventing an answer. The goal you can measure is fewer confident errors, and you can see progress on it only with a test that scores "I don't know" separately from a wrong answer.

A first baseline can be small: a few dozen questions from your own domain whose answers you already know, including several the system should decline. Run them, grade each answer correct, wrong or declined, and write down wrong answers as a share of all questions and as a share of the questions answered, beside the share declined. Grow the set before you use it to compare two systems.