E2E Testing for Chat, Voice, Phone and Video AI Agents
Run Thousands of Test Scenarios against Your Agent and Catch Hallucinations, Bias, and Toxicity Before Users Do.
AI Hallucination: Causes, Examples and How to Reduce It
Part of: LLM Testing
AI hallucination explained: its types and causes, what published rates measure, documented AI and LLM hallucination examples, and how to test and reduce it.
Last Updated on:
OVERVIEW
On 22 June 2023, a federal judge in the Southern District of New York ordered two lawyers and their law firm to pay a $5,000 penalty. Their filing contained what is now called an AI hallucination, in the court's words "non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT", and they stood by those opinions after the court asked whether the cases existed. The sanctions order in Mata v. Avianca says of one invented decision: "Its legal analysis is gibberish."
The order is careful about the cause. It says "there is nothing inherently improper about using a reliable artificial intelligence tool for assistance", and the penalty was for defending the fake cases, which the court found was done in bad faith. As of 16 October 2026, Damien Charlotin's AI Hallucination Cases database lists 2,199 cases in which a court or tribunal found, or implied, that a party relied on hallucinated material, 1,501 of them in the United States.
The tool those lawyers used did what any large language model can do: it stated something false in the same fluent, confident voice it uses for everything true. That failure is called AI hallucination.
Overview
An AI hallucination is output from an AI model that is false, or not supported by anything the model was given, yet is presented as fact in the same fluent, confident style as a correct answer. As of 16 October 2026, Damien Charlotin's AI Hallucination Cases database lists 2,199 cases in which a court or tribunal found, or implied, that a party relied on hallucinated material.
Types of AI Hallucination, by What the Output Contradicts
- Invented fact or fabricated citation: An invented fact or fabricated citation contradicts the real world, as when a brief cites court decisions that do not exist, and a lookup of the case, paper or URL exposes it.
- Unsupported by a supplied document: An unsupported claim contradicts the document, search result or record given to the model, as when a summary of a contract adds a termination clause the contract does not contain.
- Ignored instruction or self-contradiction: An ignored instruction contradicts the prompt, and a self-contradiction contradicts the model's own earlier words, as when a model says an order shipped on Monday and two turns later says it has not shipped.
Why AI Models Hallucinate and How to Reduce It
- Why do AI models hallucinate?: Language models predict the next word with no truth check, so they can produce fluent text that is statistically likely and false, and they get rare or missing facts wrong, such as birthdays, obscure titles and events after the training cutoff.
- OpenAI and Georgia Tech paper (2025): Why Language Models Hallucinate, a September 2025 paper by researchers at OpenAI and Georgia Tech, argues that models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty.
- How to reduce AI hallucination: Grounding answers in a source, letting the AI system decline when the source does not cover the question, and verifying claims before they are shown all reduce AI hallucination, though no model can be right about everything.
- Does lowering the temperature stop AI hallucinations?: No. In a Communications Medicine study published in August 2025, in which six language models read clinical cases that each contained one fabricated detail, setting the temperature to 0 gave no significant improvement, while a prompt written to reduce hallucinations lowered the mean rate from 66% to 44%.
AI Hallucination, Lying and AI Agents
- Is an AI hallucination the same as the AI lying?: No. An AI hallucination is not a lie, because lying requires intent to deceive and a language model has none, but the output is still false and needs a check against a source.
- Can an AI agent hallucinate an action?: Yes. An AI agent can report a step it never performed, such as a ticket it says it closed that is still open, so the reference has to be the observed tool calls and the ticket system itself.
- Testing AI agents for hallucinated actions: TestMu AI's Agent Assurance, which is self-serve with a 14-day free trial, invokes the real agent in staging before a release and checks an action the agent says it took against the tool calls, file changes and records it can observe, returning Pass, Fail or Unable to Verify for each acceptance criterion.
What Is an AI Hallucination?
An AI hallucination is output from an AI model that is false, or not supported by anything the model was given, yet is presented as fact in the same fluent, confident style as a correct answer. The term covers text, citations, code, speech transcripts, images and, for AI agents, reports of actions that never happened.
The confidence is what separates a hallucination from a failure you can see. The Tow Center for Digital Journalism gave eight AI search tools excerpts from news articles and asked each to name the article, its publisher, its date and its URL. Its report in Columbia Journalism Review (March 2025) says the tools "provided incorrect answers to more than 60 percent of queries", and that ChatGPT Search misidentified 134 of 200 articles, signaled a lack of confidence fifteen times and never declined to answer.
A hallucinated output always contradicts something, and naming that something is the first step toward catching it:
- The world - the output states something untrue about reality: a wrong date, a court case that was never decided, a study that was never published.
- A source it was given - the output adds to or contradicts a document, search result or database row that was in front of the model. The statement may be true somewhere else and is still unsupported here.
- Its instructions or its own earlier words - the output ignores what you asked for, or contradicts something the model said earlier in the same conversation.
A statement can also be wrong without being a hallucination, for instance when the source the model was given is itself wrong and the model reports it accurately. The guide to LLM hallucination detection separates a hallucination from a factual error and compares the methods that flag each.
AI Hallucination vs Confabulation, Errors and Lying
Confabulation is the same failure under another name. NIST's Generative AI Profile (AI 600-1, 2024) lists the risk as confabulation and notes that it is known colloquially as hallucination or fabrication. The medical word fits a little better: to confabulate is to fill a gap with invented material without meaning to deceive, while a clinical hallucination is a false perception, and a model perceives nothing.
An ordinary error is wrong without the fabrication: a miscounted total, a misread table, a typo carried over from the source. You can usually trace it to a specific input or step. A hallucination leaves no such trail, because the model produced the detail itself.
Lying needs intent. A commentary in the Harvard Kennedy School Misinformation Review (August 2025) draws the line this way: AI "lacks in intent or epistemic awareness that would allow it to recognize or prevent the generation of hallucinated content." A hallucinating model is not hiding a correct answer, so the remedy is a check against a reference.
Types of AI Hallucination, With an Example of Each
The most useful way to sort AI hallucinations is by what the output contradicts, because that names the reference you compare it against. The table below builds on two published taxonomies. A survey by Huang and co-authors in ACM Transactions on Information Systems splits hallucination into factuality (conflict with real-world facts) and faithfulness (divergence from the user's input, or from the output itself), with instruction, context and logical inconsistency as the faithfulness subtypes.
A content analysis of ChatGPT errors in Humanities and Social Sciences Communications (September 2024) went finer: it identified 8 first-level error types, subdivided into 31 second-level error types. The table uses a coarser set, chosen for testing.
| Type | What it contradicts | Example | What you compare the output against |
|---|---|---|---|
| Invented fact | The real world | An assistant states that a new space telescope took the first picture of a planet outside the solar system | A trusted reference: a known answer, an official page, a database |
| Fabricated citation or source | The real world | A brief cites court decisions that do not exist. In the Tow Center test, more than half of the responses from two AI search tools cited fabricated or broken URLs | A lookup: does the case, paper or URL exist, and does it say what is claimed |
| Unsupported by a supplied document | The document, search result or record given to the model | A summary of a contract adds a termination clause the contract does not contain | The supplied document, claim by claim |
| Ignored instruction | The prompt | Told to answer from the attached policy and nothing else, the model answers from general knowledge | The instruction and its constraints |
| Self-contradiction | The model's own earlier words | The model says an order shipped on Monday, then two turns later says it has not shipped | Earlier turns of the conversation, or repeat runs of the same question |
| Hallucinated action | The record of what an AI agent did | An agent reports that it closed a ticket, and the ticket is still open | The observed tool calls and the state of the system the agent was meant to change |
| Non-text hallucination | The audio or image the output was supposed to match | A speech-to-text model writes a sentence that nobody spoke | The original recording or image |
Read the last column before you choose a test. It tells you what you have to collect first: questions with known answers, the documents the system retrieves, the prompt, the conversation history, or access to the system an agent writes to. If you cannot name the reference for a type, you cannot yet measure that type.
Generated code has its own version of the first two rows: packages and API methods that do not exist. The guide to AI code hallucinations covers how to catch those.
Causes of AI Hallucination, and the Fix That Targets Each One
AI models hallucinate for several separate reasons, and a fix aimed at one cause does little for the others. Retrieval, for example, supplies a missing fact and does nothing about a model that ignores the passage it was handed.
| Cause | What it looks like | The fix that targets it | What that fix does not solve |
|---|---|---|---|
| Next-word prediction with no truth check | Fluent text that is statistically likely and false | Check the output against a reference after it is generated | The error is still produced, and the checker can be wrong too |
| Rare or missing facts | Wrong birthdays, obscure titles, events after the training cutoff | Retrieval: put the fact in front of the model when it answers | A retriever that returns the wrong passage, or a model that ignores the right one |
| Tests that reward a guess over "I don't know" | A model that always answers and scores well on accuracy | Score a wrong answer below a declined answer, and allow the system to decline | More declined answers, including on questions the system could have answered |
| A familiar-looking name that switches off the default refusal | Confident detail about a person or product the model half recognizes | Require a source for claims about named people and things, and test with obscure and made-up names | Half-known names that your test set did not include |
| No grounding, or grounding that is ignored | An answer from general knowledge although a document was supplied | Restrict the answer to the supplied text and check each claim against it | A stale or wrong source: a faithful answer to a wrong document is still wrong |
| Agreeing with the user | The model accepts a false premise in the question, or drops a correct answer when pushed | Test with false-premise and pushback prompts, and instruct the system to correct a wrong premise | Pressure in forms you did not test |
| Ambiguous prompts | The model fills in a missing detail, such as which product or which year, instead of asking | Have the system ask a clarifying question when a required detail is missing | Facts the model does not have |
Rows three and four come from 2025 research that changed the standard account, which used to stop at flawed training data. Why Language Models Hallucinate (Kalai, Nachum, Vempala and Zhang, of OpenAI and Georgia Tech, September 2025) argues that models hallucinate "because the training and evaluation procedures reward guessing over acknowledging uncertainty". Under a scheme that awards 1 point for a correct answer and none for "I don't know", a model that always guesses outscores one that admits doubt.
The same paper puts a floor under the rare-facts row: "if 20% of birthday facts appear exactly once in the pretraining data, then one expects base models to hallucinate on at least 20% of birthday facts."
Anthropic's interpretability research, Tracing the thoughts of a large language model (March 2025), looked inside a Claude model and found that "refusal to answer is the default behavior": a circuit that says the model lacks the information stays on unless a "known entities" feature switches it off. A hallucination can follow when the model "recognizes a name but doesn't know anything else about that person". By switching on the "known answer" features for a name the model did not know, the researchers write, "we're able to cause the model to hallucinate (quite consistently!) that Michael Batkin plays chess."
Both findings change what you test. Put half-known names and unanswerable questions in your test set, and score a declined answer differently from a wrong answer, because a test that counts correct answers alone rewards the guessing you want to remove. The MMLU benchmark counts correct answers alone, scoring the share of its multiple-choice questions answered with the keyed letter.
Why Published Hallucination Figures Disagree
Published hallucination figures rarely agree, because each one is a count of one thing divided by a count of another, on one task, judged by one grader. Per-model figures from the main benchmarks, each with what it counts, are in the guide to AI hallucination rate.
The figures disagree for reasons you can check in any report:
- Different tasks - answering from memory, citing a news article and researching case law are different jobs. Vectara's hallucination leaderboard measures a fourth: summarizing a document the model was handed.
- Different handling of "I don't know" - a declined answer can count as wrong, count as neutral or drop out of the denominator altogether.
- Different graders - a match against a reference answer, a detector model and a panel of journalists will not agree on what counts as a hallucination. Grader choice matters on other benchmarks too, as the guide to the GDPval benchmark shows by setting expert graders against an LLM judge.
The guide to LLM benchmarks covers what public leaderboards measure in general. To show the second reason on its own, a short script was run on 7 October 2026 with Node.js v25.5.0. Its input is sample data: two imaginary systems, the same 20 questions, and a grade for each answer (correct, wrong or abstained) written by hand.
node rates.mjs20 questions, two sample systems
System A (answers every question)
System B (abstains when unsure)
System A System B
----------------------------------------------------------
Correct answers 13 12
Wrong answers 7 2
Abstained (said it did not know) 0 6
Accuracy: correct / all questions 65.0% 60.0%
Wrong answers / all questions 35.0% 10.0%
Wrong answers / questions it answered 35.0% 14.3%
Abstentions / all questions 0.0% 30.0%
Score if a wrong answer costs a point 6 10
Ranked by accuracy alone: System A first
Ranked by fewest wrong answers: System B first
Ranked with wrong answers penalized: System B firstThe sample set measures nothing about any real model, and 20 questions is far too few for a real comparison. It shows how the counting rule changes the headline number:
- System A has the better accuracy (65.0% against 60.0%) and gives 7 wrong answers where System B gives 2.
- For System B, wrong answers over all questions is 10.0% and wrong answers over the questions it answered is 14.3%. Both are honest rates from the same 20 grades.
- Ranked by accuracy alone, the system that guesses comes first. Under either rule that counts or penalizes wrong answers, the system that abstains comes first.
OpenAI's post Why language models hallucinate (5 September 2025) shows the same effect in published SimpleQA data. By its figures, o4-mini had the slightly higher accuracy (24% against 22%) and nearly three times the error rate (75% against 26%), because it abstained on 1% of questions where gpt-5-thinking-mini abstained on 52%. The post draws the conclusion itself: "Strategically guessing when uncertain improves accuracy but increases errors and hallucinations."
Before you compare two published rates, find the numerator, the denominator and what happened to the declined answers. To report a rate for your own chatbot, the hallucination rate formula and its denominator are set out in our guide to chatbot metrics.
Real AI and LLM Hallucination Examples and Their Consequences
Fabricated court citations, as in the case that opens this guide, are documented decision by decision in Charlotin's database. The table lists AI and LLM hallucination examples from 2022 to 2025, each dated by the court record, paper or first report it comes from. Rows whose source is linked below are told in more detail after the table, with what each source does and does not establish.
| Date | System | What it stated | Consequence | Source |
|---|---|---|---|---|
| Nov 2022 | Meta's Galactica, a language model for science, in a public demo | Fake scientific papers, some credited to real authors, and wiki articles on the history of bears in space | Meta paused the public demo within three days of unveiling it | MIT Technology Review, 18 November 2022 |
| Feb 2023 | Google's Bard chatbot, in Google's own launch material | That the James Webb Space Telescope took the very first pictures of a planet outside our solar system; the first such image dates from 2004 | Alphabet shares slid 9% in trading on 8 February, amid investor fears about Microsoft's rival search product | The Verge and The Guardian, linked below |
| Feb 2023 | Microsoft's Bing AI chat, in its launch demo | A summary of Gap's quarterly earnings report with a 5.9% operating margin the report does not contain, and a forecast of sales growth where the report forecast a decline | A researcher found the errors by checking the report; Microsoft said the system may make mistakes during its preview period | Dmitri Brereton's analysis, 13 February 2023, and CNBC's report of 14 February 2023 |
| Jun 2023 | ChatGPT, used for legal research in Mata v. Avianca | Court decisions, with quotes and citations, that do not exist | A $5,000 penalty for two lawyers and their firm, who stood by the fake opinions after the court questioned them | The court's sanctions order, linked in the opening |
| 2024 | OpenAI's Whisper speech-to-text model | Whole phrases and sentences nobody spoke, in roughly 1% of the transcriptions studied | No fine or ruling; the Associated Press found a Whisper-based tool, used by over 30,000 clinicians, that erases the original audio | Careless Whisper and the Associated Press, below |
| Dec 2024 | Apple Intelligence notification summaries | An alert that made it appear BBC News had reported that Luigi Mangione shot himself; he had not | The BBC complained to Apple; iOS 18.3, released in January 2025, made summaries for news and entertainment apps temporarily unavailable | BBC News, 13 December 2024; Apple's iOS 18 release notes and its iOS 18.3 security notes |
| Mar 2025 | ChatGPT | That a real Norwegian man had been convicted of murdering two of his children, mixed with true details of his life | A complaint to Norway's data protection authority, with no decision reported | noyb, linked below |
| May 2025 | ChatGPT | That radio host Mark Walters was the accused embezzler in a lawsuit it was asked to summarize | A Georgia court granted OpenAI summary judgment; Walters testified he incurred no damages | The court's order, linked below |
| May 2025 | An AI tool a freelance writer used for a newspaper insert | A summer reading list on which 10 of 15 titles were fake books credited to real authors | The Chicago Sun-Times took the section off its e-paper, and King Features, which supplied it, ended its work with the writer | Chicago Sun-Times review, 29 May 2025 |
| Aug 2025 | A Deloitte report for the Australian government, produced with help from generative AI | References to academic reports that do not exist and a made-up reference to a court decision | Deloitte agreed in October 2025 to a partial refund on the A$440,000 report | The Guardian, linked below |
Speech-to-text in hospitals (2024) - the paper Careless Whisper, presented at ACM FAccT 2024, tested OpenAI's Whisper on recordings from TalkBank's AphasiaBank in April and May 2023. It found that "roughly 1% of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio", and that 38% of those hallucinations included explicit harms such as perpetuating violence or implying false authority.
An Associated Press investigation published on 26 October 2024 reported that more than 30,000 clinicians and 40 health systems had started using a Whisper-based tool to transcribe patient visits, and that the tool erased the original audio. The AP report describes no fine or ruling, and it noted OpenAI's own warning against using Whisper in "high-risk domains".
A consulting report for a government (2025) - The Guardian reported on 6 October 2025 that Deloitte would give the Australian government a partial refund "over a $440,000 report that contained several errors, after admitting it used generative artificial intelligence to help produce it". That figure is the value of the report in Australian dollars, and the refund amount had not been made public when the story ran.
The errors, reported by the Australian Financial Review in August 2025, included references to academic reports that do not exist and a made-up reference to a court decision. Deloitte "did not state that artificial intelligence was the reason behind the errors"; the University of Sydney academic quoted by The Guardian, Dr Christopher Rudge, described them as hallucinations.
A product launch (2023) - in Google's own promotional material for its Bard chatbot, Bard said the James Webb Space Telescope "took the very first pictures of a planet outside of our own solar system". The Verge reported on 8 February 2023 that the first image of an exoplanet was taken in 2004.
The Guardian reported the next day that Alphabet stock "slid by 9% during regular trading in the US". The fall followed the error and, by the same report, came amid investor fears about Microsoft's competing search product. Google's statement to The Verge named the remedy: "a rigorous testing process" so that responses "meet a high bar for quality, safety and groundedness in real-world information."
A false accusation against a real person (2025) - on 20 March 2025 the privacy group noyb filed a complaint against OpenAI with Norway's data protection authority. Asked about a Norwegian man, ChatGPT had presented him as a convicted criminal who murdered two of his children, mixed with real details such as the number of his children and the name of his home town. That is a complaint, not a ruling, and noyb's page reported no decision on it as of 16 October 2026.
In the United States, a Georgia court granted summary judgment to OpenAI on 19 May 2025 in Walters v. OpenAI, a defamation claim over a ChatGPT output that named a radio host as the accused embezzler in a lawsuit it was asked to summarize. The order records that Walters "testified that he incurred no damages".
In most rows of the table, a reference that could have exposed the error already existed: the cited works, a NASA page, Gap's own report, the BBC's actual story, the recording, a book catalog. The check was skipped, or in the transcription tool's case the recording had been erased.
How to Test an AI System for Hallucinations
A hallucination test compares an answer with a reference and records one of three outcomes. Its parts are the same whatever the system:
- Input - a question with a known answer, or a question together with the source the system must answer from.
- Output - the system's answer, captured exactly, and captured on several runs because the same question can produce different answers.
- Reference check - the answer compared with the known answer or, claim by claim, with the source.
- Outcome - correct, wrong or declined.
Score "declined" as its own outcome. If a declined answer counts as wrong, the test pushes you toward a system that always answers, and the sample output above shows what that hides: the system that guessed looked better on accuracy and gave more than three times as many wrong answers. If declined answers are left out altogether, a system can look safe by answering almost nothing.
Published benchmarks that separate the outcomes work the same way. In SimpleQA, the question set behind the first row of the rates table, each answer "is graded as either correct, incorrect, or not attempted." Groundedness scoring and judge models automate the reference check at scale, and each fails in its own way, which the LLM hallucination detection guide compares. How guesses and abstentions change a score is worked through in the SimpleQA benchmark guide.
For the step-by-step version on a deployed support bot, from building the question set out of your own knowledge base to deciding which results block a release, see the guide to chatbot hallucination, which also splits "declined" into a correct refusal and an over-refusal. Hallucination tests are one part of a wider LLM testing plan, next to regression, safety and performance tests.
The Agent Testing Certification by TestMu AI covers the Agent Testing platform, which tests chat, voice, phone, video and image agents, along with Agent Assurance and Rook CLI.
Hallucinated Actions: Checking What an AI Agent Did With Agent Assurance
Most of the types above are text you can check against other text: a reference, a supplied document, the prompt or the conversation. An agent with tools can also hallucinate an action: its final message says a record was updated or a message sent, and the step never ran. A 2025 survey of hallucinations in LLM-based agents defines execution hallucinations as cases where agents "claim to have completed certain sub-stages during the execution phase, but in reality, they have not actually been performed or accomplished", one of five types of agent hallucinations it classifies.
Take an agent that works an engineering issue tracker and is asked to close a resolved ticket. It replies that the ticket is closed, and the ticket is still open because the close call returned an error the agent did not act on. The reply, the transcript and any score computed from them all rest on the agent's own account, so the reference has to be the observed tool calls and the ticket system itself. The guide to agent action hallucination breaks failure modes like this one down at the trace level.
TestMu AI's Agent Assurance invokes the real agent in staging before a release and decides each acceptance criterion on evidence. For an action the agent says it took, that works like this:
- Tool calls - the calls your profile reports the agent made during the run, checked against the list of tools the agent declares and against any it must never call.
- Files and artifacts - changes on disk inside the paths your profile lists, plus any artifacts the run created.
- Records - if the agent says it created or changed a record, a read-only lookup tool that you supply and approve checks that the record exists and holds the right values.
- Hallucination scenarios - hallucination is one of nine adversarial scenario categories. The adversarial class is generated by default, and you can narrow a suite to the classes or categories you want.
- Verdicts - the possible results per criterion are Pass, Fail and Unable to Verify. Unable to Verify means the evidence could not establish the result, so the pass rate leaves it out instead of counting it as a failure.
How much Agent Assurance can observe depends on the profile and the access you give it. A profile that returns only the agent's answer leaves most action criteria Unable to Verify.
Eval and observability tools mostly grade the agent's words and the traces it logged, while Agent Assurance looks at what a run changed and names the criteria it could not check. That describes where each kind of tool usually looks, and an eval tool is still the right check for an invented fact inside an answer.
Agent Assurance tests before a release and does not watch live traffic, so it is neither a guardrail nor a hallucination detector for production. Whether to verify an agent before release, at runtime or after each run is weighed in the guide to AI agent verification.
From a terminal, Agent Assurance runs as Rook CLI, and it is self-serve, with a 14-day free trial that includes 4,000 credits. Install it with npm, then check the version:
npm install -g @testmuai/rook@latest
rook --versionYou can install Rook CLI on macOS and Linux, and 64-bit Windows through npm or WSL. Its npm package needs Node.js 20 or newer. In Claude Code, the command below adds the rook skill; it needs Node.js 22 or newer and installs the skill only, not the CLI:
npx @testmuai/rook-skill@latest install --agent claude-codeThe Agent Assurance overview sets out which evidence counts and how each verdict is reached. Aim every run at staging, since anything your agent writes during a test is a real write.
How to Reduce AI Hallucinations, and Whether They Can Be Eliminated
Each method below targets a cause from the causes table, and the published measurements show how large an improvement to expect.
For a quick visual walkthrough of this concept, watch the video below from TestMu AI.
Subscribe to the TestMu AI YouTube channel for more tutorials on testing and automation.
- Ground the answer in a source, then check the answer against it - retrieval puts the missing fact in the prompt, and a claim-by-claim check catches the model ignoring it. RAG evaluation metrics such as faithfulness measure that second step. Among the Ragas metrics, faithfulness is the share of a response's claims that the retrieved context supports.
- Let the system decline, and stop penalizing it for declining - tell it to say so when the source does not cover the question, and score a declined answer as its own outcome.
- Verify claims before they are shown - in Chain-of-Verification (Dhuliawala and co-authors, arXiv 2309.11495), the model drafts an answer, plans verification questions, answers them independently and then revises. On a Wikidata list task this more than doubled precision over the Llama 65B few-shot baseline, "from 0.17 to 0.36". That is a 2023 model, and at 0.36 most listed items were still wrong.
- Narrow the scope - a system limited to one product's documentation has fewer half-known names to guess about than a general assistant. Scope and output rules are one kind of AI guardrails.
- Keep a person in the loop where a wrong answer is costly - legal filings and published reports both appear earlier in this guide, and each passed through someone who did not check; in the transcription case the recording a reviewer would have needed had been erased.
- Test on every change - a new model version, a prompt edit or a knowledge-base update can move the rate, so rerun the same question set and compare.
Prompt instructions and a low temperature do less than expected. In a study in Communications Medicine (August 2025), six language models read 300 physician-validated clinical vignettes that each contained one fabricated detail, and a prompt written to reduce hallucinations lowered the mean rate from 66% to 44%, a real gain that still left the models elaborating on the invented detail in 44% of outputs. On temperature, the study reports: "Temperature adjustments offer no significant improvement."
Whether AI hallucination can be eliminated depends on the definition. Xu, Jain and Kankanhalli (arXiv 2401.11817, January 2024) define hallucination formally as any inconsistency between a model and a ground-truth function, and show that under that definition "it is impossible to eliminate hallucination in LLMs" used as general problem solvers. OpenAI's 2025 post agrees that accuracy "will never reach 100%" because some questions are unanswerable, and still rejects the claim that hallucinations are inevitable: "They are not, because language models can abstain when uncertain."
The two positions fit together. No model can be right about everything, and a system can be built and graded so that it says it does not know instead of inventing an answer. The goal you can measure is fewer confident errors, and you can see progress on it only with a test that scores "I don't know" separately from a wrong answer.
A first baseline can be small: a few dozen questions from your own domain whose answers you already know, including several the system should decline. Run them, grade each answer correct, wrong or declined, and write down wrong answers as a share of all questions and as a share of the questions answered, beside the share declined. Grow the set before you use it to compare two systems.
Author
Anubhav Singhmaar is an AI Product Manager at TestMu AI driving Kane CLI, the command-line tool that brings browser automation to the terminal, turning natural-language flows into runs in a real Chrome browser that return pass or fail with shareable proof. He owns the roadmap and prioritization and works with engineering to ship developer-facing features. Before TestMu AI, he spent over four years at Sprinklr owning enterprise voice AI across APAC and EMEA. A mechanical engineer turned product manager, he grounds guidance in real QA workflows.
Reviewer
Samyak Goyal is a Senior Member of Technical Staff at TestMu AI engineering Kane CLI, the command-line tool that runs browser automation from the terminal, where a flow described in natural language executes in a real Chrome browser and returns pass or fail with shareable proof. He is a backend engineer with 4+ years of experience, previously an SDE at Innovaccer, where he built APIs, introduced Kafka, and cut deployment from weeks to hours. Samyak also builds multi-agent systems, skill-orchestration frameworks, and a personal copilot that indexes 200+ microservice repositories.
AI Hallucination FAQs
Did you find this page helpful?
More Related Learning Hubs
TestMu AI forEnterprise
Get access to solutions built on Enterprise
grade security, privacy, & compliance
- Advanced access controls
- Advanced data retention rules
- Advanced Local Testing
- Premium Support options
- Early access to beta features
- Private Slack Channel
- Unlimited Manual Accessibility DevTools Tests




















