Hallucinations are among the most important limitations of today's language models. The tricky part is that false statements are phrased just as confidently as correct ones. A model might cite a legal provision that actually governs something entirely different, refer to a study that does not exist, or describe a software feature that never existed.
The term is meant figuratively. The model does not perceive anything; it generates a text that is statistically plausible but does not correspond to reality.
How do hallucinations arise?
A large language model generates text by choosing likely continuations step by step. In doing so, it does not check its statements against a database of verified facts. If the model lacks knowledge, if that knowledge is outdated, or if the question is ambiguous, a fluent answer is produced anyway, just a wrong one.
Hallucinations are encouraged by, among other things:
- questions on very specialized, rare, or current topics
- requests for precise details such as numbers, quotes, sources, or case numbers
- missing context in the request
- tasks in which the model is pushed toward an answer even though no meaningful one is possible
Typical examples
- A summary contains a number that does not appear in the source document.
- An assistant cites a court ruling or a standard that does not exist in that form.
- A customer service reply promises a service that the company does not offer at all.
- A code suggestion uses a function that does not exist in the library being used.
What companies should keep in mind
- Tie answers to sources: With retrieval-augmented generation, the model relies on your own verified documents and can cite references. This lowers the risk but does not rule out errors.
- Plan for human review: The greater the impact, the more important approval by experts becomes, see Human in the Loop.
- Allow uncertainty: Design prompts so that the model openly states missing information instead of guessing.
- Separate tasks: Make decisions using fixed, traceable rules, and use AI where it phrases or summarizes.
- Train employees: AI literacy includes knowing that AI outputs can be wrong and how to check them.
How TOMORIS relates
In DarkWatch, the leak monitoring service from TOMORIS, rules decide and the AI phrases – an example of this separation of tasks.
View AI training