Hallucination
When an AI model produces fluent, confident output that is false or unsupported by its sources. What matters is which errors are unacceptable in a given workflow and how they are detected.
A hallucination is output from a generative AI model that sounds plausible and is wrong, or cannot be traced to any supporting evidence. A language model predicts likely text; it does not check claims against reality unless the system around it is built to do so.
"Does AI hallucinate?" is one of the wrong questions about AI, because the answer is always yes, sometimes. The useful version is specific to a workflow:
- Which errors are unacceptable here?
- What evidence must the system provide for its claims?
- How will unsupported output be detected?
- What is the cost of a false positive versus a false negative?
Reducing and containing it
Grounding the model in retrieved sources with retrieval-augmented generation helps, but does not eliminate the problem, because the model can still misread or ignore what it retrieves. Add guardrails, require citations, route uncertain cases to a person, and track how often people have to repair a recurring error; that repair work shows up in supervision load. Good calibration helps a system know when it is not sure.
Read more in The wrong questions about AI right now.
Related terms
RAG
Retrieval-augmented generation: a pattern where a model is given relevant documents or data retrieved at question time, so its answers rest on current, specific sources rather than only its training.
Guardrails
Controls placed around an AI system — input and output checks, permission limits, review steps and escalation rules — that keep its behaviour within acceptable bounds.
Calibration
How well a model's stated confidence matches reality: if it says 90% on a hundred comparable cases, roughly ninety should be correct. Once probabilities drive actions, it becomes an operating metric.
Used in these essays
The wrong questions about AI right now
Many of the questions that helped us orient ourselves around generative AI are now too blunt to be useful. The harder work is no longer asking what AI is in the abstract, but specifying where it works, where it fails, what authority it should have, and what the whole system costs.
The hidden metric in AI automation is supervision
As AI moves from assisting work to leading it, hours saved stop telling the whole story. The scarce resource shifts to human supervision: approvals, exceptions, context and judgment.