Why AI hallucinates, and how to catch it

The fear was never that you couldn't use it. Anyone can write a prompt now. The quiet worry is different: looking exposed because you trusted an answer that turned out to be wrong. The old question was whether you could use the tool at all. The real question is whether you can catch it when it is confidently wrong.

The behaviour has a name: hallucination. AI does not only answer from things it knows. It is built to produce the most plausible-sounding text, whether or not the fact behind it is real, so it fills gaps smoothly instead of stopping to say it is not sure. The result reads clean and can still be invented.

Confidence is not a signal here. Research on this is unambiguous: a model can state something with full certainty and still be wrong, even on a fact it actually holds (Simhi et al., 2025, "Trust Me, I'm Wrong"). The hesitation you would read in a person is missing. A smooth, assured answer is not evidence that it is right.

Nor is picking the smartest model. The hallucination rates people quote come from short, tidy news articles, where leading models sit around 2 percent. On the longer real-world documents you actually work with, leading reasoning models cross 10 percent. Being the most capable model did not make it the most faithful. One honest caveat: rates depend heavily on the model, the task and the document, so read these as direction rather than a fixed score.

Two habits protect you. First, verify the load-bearing facts: not the whole output, the two or three facts your decision actually rests on, checked against the source. Second, do not relax because you picked the smart model. On hard, real documents the capable models fail more, not less. Capability is not the same as faithfulness.

The value is moving. Producing a draft is cheap now; anyone can do it. The scarce skill is judgment: knowing what good looks like, and catching the confident answer that is quietly wrong.