Definition
Language models generate the most plausible continuation of text, not verified facts. When they lack the information, they can fill the gap with something that sounds right. NIST's generative AI risk profile calls this 'confabulation' and lists it among the main risks of the technology.
Hallucination cannot be removed entirely, but it can be made rare and catchable: retrieve the right sources, require citations, check outputs against systems of record, and design the process so that unsupported answers are flagged or routed to a person.