Hallucination
How it works
Hallucination arises at the intersection of data, training and inference. (1) Training data contain errors, contradictions and gaps, and the model learns a probability distribution over text, not a fact base. (2) The training objective (next-token prediction) optimizes fluency, not verifiability; fine-tuning and RLHF can additionally reward confident, helpful answers at the expense of caution. (3) At inference, stochastic sampling (temperature, top-k, top-p) increases diversity but also the risk of generating untrue details; the model has no built-in fact-checking mechanism and no uncertainty signal calibrated to truth. As a result the model 'fills gaps' with statistically plausible but fabricated text (confabulation). Mitigation includes grounding in sources (RAG), constrained decoding, chain-of-thought and self-consistency, verifiers, and changing how evaluations are scored so that guessing is not rewarded.
Problem solved
It names and structures a core trustworthiness limitation of generative models: the fact that fluency and confidence do not guarantee truthfulness. It provides the concepts (faithfulness vs factuality, intrinsic vs extrinsic) needed to measure, diagnose and mitigate generation errors in high-stakes applications.
Components
Generated content directly contradicts the provided context or source material.
Generated content cannot be verified against the source — it is neither supported nor contradicted by the context.
Evaluation axis: whether the output is consistent with the provided source material (regardless of world truth).
Evaluation axis: whether the output agrees with world facts (regardless of the source).
Alternative term emphasizing that the model confidently fills gaps with fabricated but plausible-sounding details, without intent to deceive.
Official
Implementation
A smooth, confident tone leads users to trust fabricated content.
Even with a retriever the model may ignore or misread documents (intrinsic hallucination).
Models generate plausible-looking but nonexistent sources, DOIs or court cases.
Scoring that rewards confident answers entrenches hallucination at the expense of admitting uncertainty.
Models poorly recognize their own hallucinations without external knowledge or extra reasoning steps.
Evolution
Early uses of the term for content unsupported by the input in NMT and captioning.
Work on the faithfulness of summaries to the source document formalizes the faithfulness axis.
Benchmark shows that larger models can imitate human misconceptions (inverse scaling).
The survey defines a taxonomy (intrinsic/extrinsic, faithfulness/factuality) and organizes the field.
The spread of LLM chatbots makes hallucination a central trustworthiness problem; dedicated benchmarks appear.
The paper argues that benchmarks reward confident guessing, which entrenches hallucinations.
Hyperparameters (configurable axes)
Higher temperature and more stochastic sampling increase diversity but also the tendency to fabricate details.
Providing the model with verified source documents lowers hallucination but does not eliminate it.
The fine-tuning regime affects calibration and the tendency to guess confidently rather than acknowledge uncertainty.
Larger models tend to make fewer factual errors, but scale does not remove hallucination and can be misleading (e.g. imitating human misconceptions).
Clear instructions, requesting citations, and allowing the model to say 'I don't know' reduce hallucinations.