In 2023, two lawyers submitted a legal brief in a federal case—Mata v. Avianca—that cited over half a dozen court decisions. The citations looked authoritative: case names, docket numbers, dates, judicial reasoning. The judge requested copies of the actual decisions. None of them existed. The lawyers had used ChatGPT to research case law, and the model had invented the citations entirely—names, docket numbers, and legal arguments included.
The lawyers were sanctioned. The judge was not amused. And the story became a cautionary tale that traveled well beyond the legal profession.
This is what AI researchers call a hallucination—and it's worth understanding how it happens, because it affects anyone using AI to answer questions based on specific facts.
1. What a Hallucination Actually Is
The term is a bit dramatic. An AI model doesn't hallucinate the way a person does. But the name captures something real: the model produces output that sounds coherent and confident while being factually wrong.
What makes this particularly tricky is the confidence. The model doesn't say "I'm not sure about this." It states incorrect information in the same tone it uses for things that are accurate. A fabricated court case reads like a real one. An invented statistic reads like a cited one. There are usually no visible seams.
A more precise definition: a hallucination is when a language model generates text that appears grounded in fact, but isn't verifiable against any real source.
2. Why It Happens: The Token Prediction Problem
To understand hallucinations, you need to understand what language models actually do—because it's not what most people assume.
Language models are trained to predict the next token—roughly a word or part of a word. Given a sequence of text, the model learns: what would typically come next in this kind of text? It learns this by training on enormous amounts of human writing.
The result is a model that's extraordinarily good at producing text that sounds like human writing. But there's no separate fact-checking step. The model doesn't have a database of verified facts it queries before responding. It has learned statistical associations—patterns between words and concepts.
When you ask it "What's the refund policy for Acme Store?", it doesn't look anything up. It predicts: given this question, what would a plausible answer look like? If Acme's policy isn't in its training data or provided context, it generates an answer that sounds like a refund policy. The answer might be wrong in every specific detail while sounding completely reasonable.
This is the root cause: language models generate plausible text, not verified text.
3. The Main Types of Hallucinations
Not all hallucinations look the same. A few patterns come up consistently:
Factual confabulation is the most common. The model invents plausible-sounding facts—dates, names, version numbers, or statistics that are close to correct but subtly wrong, or entirely made up. The closer a wrong answer is to being right, the harder it is to catch.
Citation hallucination is what caught those lawyers. The model invents research papers, court cases, or articles that don't exist—often with convincing author names, journal titles, and publication years. The structure looks right; the content is fabricated.
Temporal errors occur because language models have a knowledge cutoff—they don't know about events after their training data ends. But they don't always flag this limitation. A model might answer questions about recent events by pattern-matching from older information, with no indication that it's working from potentially outdated data.
Specificity inversions are where the model gets the general concept right but the specific detail wrong. The kind of error that passes a quick skim but fails on closer inspection.
4. Why Some Topics Are Riskier Than Others
Hallucination risk isn't uniform. It correlates with how well-represented a topic is in training data.
Popular topics—well-documented programming languages, widely-covered historical events, established scientific concepts—tend to produce more reliable responses. The model has seen many consistent examples and learned accurate patterns.
Edge cases are where things break down: niche regulations, internal company policies, obscure technical specifications, recent events, or specialized professional knowledge. The model has few examples to draw from, so it pattern-matches from adjacent concepts—and gets things wrong.
The practical implication: the more specific the question, the higher the hallucination risk when the model is working from general knowledge alone. Questions about your product's specific pricing tiers, your refund process, or your integration limitations carry much more risk than generic questions about your industry.
5. Five Ways to Reduce Hallucinations
The good news is that hallucinations aren't inevitable—several techniques can meaningfully reduce them.
Retrieval-Augmented Generation (RAG) is currently the most widely adopted approach for grounding AI in verified content. Instead of relying on training data, the model first retrieves relevant documents from a curated knowledge base, then generates a response based on those documents. When the answer is right there in a retrieved source, the model has much less reason to invent an alternative. Studies in this area suggest RAG can meaningfully reduce factual errors compared to purely generative approaches, though results vary by domain and knowledge base quality.
Explicit grounding instructions can also help. Instructing the model to only answer based on provided context—and to say "I don't know" when it can't find an answer—doesn't eliminate hallucinations, but it meaningfully reduces confident fabrication. The model needs permission to admit uncertainty; by default, it tends to fill gaps.
Requiring source citations creates a natural forcing function. If the model must point to a specific document for every claim, it becomes much harder to invent unsupported answers. This serves double duty in support contexts: citations reduce hallucinations and build user trust by making answers verifiable.
Lower temperature settings reduce how "creative" the model is in generating responses. Temperature is a parameter that controls randomness—higher values produce more diverse output, lower values produce more conservative, predictable output. For factual Q&A, lower temperature generally produces more grounded responses.
Scope restrictions limit what the model will respond to. A support chatbot configured to only answer questions about your product has far less surface area for hallucination than one that will discuss anything. Out-of-scope questions should route to humans, not generate pattern-matched guesses.
6. How to Choose the Right Hallucination Controls
None of these techniques—RAG, grounding instructions, citation requirements, temperature tuning, or scope restrictions—eliminates hallucinations entirely. The goal is risk management, not elimination.
A useful frame: match the mitigation level to the consequences of a wrong answer. For a customer support chatbot answering questions grounded in your documentation, RAG with citations and tight scope restrictions can reduce risk to acceptable levels for most use cases—the consequence of an occasional wrong answer is a follow-up question or a handoff to a human.
For applications where errors carry serious consequences—medical guidance, legal research, financial advice—much stricter controls are warranted. Human review of AI output, strict factual grounding, or avoiding AI-generated responses entirely may be the right call. The Mata v. Avianca case is a useful reminder that hallucinations don't announce themselves. In high-stakes domains, treating AI output as "probably right until proven wrong" isn't a safe default.
Final Thoughts
Understanding hallucinations isn't a reason to distrust AI—it's a reason to deploy it thoughtfully. Just as you'd verify a new tool before fully relying on it, we'd suggest building verification into any AI deployment until you've established its reliability in your specific context.
RAG, grounding instructions, and scope restrictions are actively used in production systems to make AI outputs more reliable. Newer models also tend to hallucinate less than their predecessors, and research into better grounding methods is ongoing. If you're exploring how to ship a chatbot without losing the human touch, understanding hallucination risk is a good foundation.
For most practical applications, the question isn't "will this AI ever hallucinate?" It's "have we reduced the risk enough given what's at stake?" That's a question worth asking carefully before deploying—and one the mitigations in this piece can help answer.