Hallucination
confabulation · fabrication
Kurz gesagt
A hallucination is output that is fluent and confident but factually wrong or unsupported — an invented citation, a non-existent function, a plausible number with no source. It happens because the model generates statistically likely continuations, and a likely-sounding claim is not necessarily a true one.
The word is imprecise — the model is not perceiving anything — but it has stuck. What it describes is real and predictable: output that reads like a well-sourced answer and is not one.
The mechanism is straightforward. A language model produces likely continuations of text. "The relevant provision is Article 14(3)" is a highly likely shape of sentence in a legal answer, whether or not Article 14(3) says anything of the kind. The model has no separate step that checks the claim against the world.
Certain conditions raise the rate reliably: questions about specific identifiers, dates or citations; topics thin in the training data; requests that presuppose something false; and any prompt that makes admitting ignorance feel like a failure to be helpful.
The mitigations that work are architectural rather than verbal:
- Ground the answer. Retrieve source passages and instruct the model to answer only from them.
- Require citations at claim level, rendered as links a reader can check.
- Permit refusal explicitly. "If the sources do not contain the answer, say so."
- Verify mechanically where possible — run the code, resolve the URL, check the identifier exists.
- Measure it. Track the rate on a fixed evaluation set instead of relying on impressions.
None of this makes a model truthful. It makes the wrong answers detectable, which is a different and achievable goal.
Häufige Fragen
- Can hallucination be eliminated?
- Not entirely with current architectures. It can be substantially reduced by grounding answers in retrieved sources, requiring citations, and explicitly permitting the model to say it does not know.
- Why does the model sound so certain when it is wrong?
- Fluency and factuality are produced by the same mechanism. Nothing in the generation process tracks confidence in a claim, so tone carries no information about reliability.