Definition
An AI hallucination is output from a generative AI model that is presented fluently and confidently but is false, fabricated or not supported by the information the model was given. Hallucinations range from small errors — a wrong date, a misquoted figure — to entirely invented court cases, research papers, product features or biographical details. The word is borrowed loosely from psychology: the model "perceives" something that is not there and reports it as fact.
Hallucinations are not bugs in the traditional sense. They are a side effect of how large language models work. An LLM generates text by predicting the most plausible next token, not by looking facts up in a verified database. When the model lacks the right information, it still produces something that looks like a good answer, because looking like a good answer is exactly what it was trained to do.
Why hallucinations happen
- Prediction, not retrieval. The model optimises for plausible language. A plausible-sounding citation and a real one look similar to it.
- Gaps in training data. Niche topics, recent events and company-specific facts may be missing, so the model fills the gap.
- Ambiguous prompts. Vague questions invite the model to assume details.
- Pressure to answer. Models are tuned to be helpful and often answer rather than say "I don't know".
- Long or complex reasoning. Errors compound across many steps, especially in calculations.
Rates vary widely by model, task and topic. Independent benchmarks such as Vectara's hallucination leaderboard show that even leading models introduce unsupported content in a small percentage of summaries, and rates are much higher for open-ended factual questions about obscure topics. Newer models hallucinate less, but none hallucinate never.
Real-world consequences
The most famous case is Mata v. Avianca in the US District Court for the Southern District of New York. The plaintiff's lawyers filed a brief citing non-existent court opinions, with fake quotes and citations, generated by ChatGPT — and kept standing by them after the court questioned them. On 22 June 2023 Judge P. Kevin Castel sanctioned the two lawyers and their firm and ordered them to pay a $5,000 penalty. Since then, courts in several countries have issued guidance requiring lawyers to verify AI-generated citations. In Moffatt v. Air Canada, 2024 BCCRT 149, decided on 14 February 2024, British Columbia's Civil Resolution Tribunal rejected Air Canada's argument that it was not liable for its website chatbot, which had wrongly told a grieving customer he could claim a bereavement fare retroactively. The airline was ordered to compensate him.
At work, the risks are usually quieter but still costly: a wrong statistic in a board deck, an invented clause in a contract summary, an incorrect policy answer given to an employee, or a fabricated source in a client report. Each damages credibility and can create legal exposure.
Example
A marketing analyst asks an AI assistant: "Give me five statistics about remote-work productivity with sources." The assistant returns five neat bullet points, each with a percentage and a citation to a well-known consultancy or university. Two of the statistics are real. One is real but attributed to the wrong organisation. Two are invented, and their links lead nowhere.
A better workflow looks like this. The analyst uses an AI tool with web search enabled, asks it to "quote the exact sentence and link for each statistic, and say 'not found' if you cannot verify one", then opens every link before using a number. Alternatively, she pastes in the three reports her team already trusts and asks the model to extract statistics only from those documents. Both approaches dramatically cut the risk because the model is grounded in real sources and the human checks the final output.
How to reduce hallucinations
- Ground the model in sources. Provide the documents, or use tools with search or retrieval-augmented generation (RAG).
- Ask for evidence. Request quotes, page numbers and links, then check them.
- Give permission to be uncertain. Add "If you are not sure, say so" to your prompt.
- Narrow the task. Specific questions produce fewer inventions than open-ended ones.
- Use tools for maths. Let the model run calculations in a spreadsheet or code interpreter.
- Keep a human in the loop. Anything that reaches customers, managers, regulators or courts needs human verification.
- Match checking to risk. A brainstorm needs little checking; a legal or financial claim needs a lot.
Hallucination versus bias versus error
A hallucination is fabricated or unsupported content. Bias is a systematic skew, such as stereotyped assumptions about gender or nationality. A simple error is a mistake on information the model did have, such as misreading a table. All three require review, but hallucinations are especially dangerous because they are delivered with the same confidence as correct answers.
Key takeaways
- Hallucinations are confident, plausible and wrong — a natural side effect of next-token prediction.
- They have caused real legal and reputational damage.
- Grounding, specific prompts, evidence requests and human review reduce them sharply.
- The person who uses AI output remains responsible for its accuracy.