If you are deciding whether to let an AI answer your team's questions, you will keep hearing the word hallucination. Here is what it means, without the jargon.
The short definition
An AI chatbot hallucination is an answer that sounds confident and is false or not supported by anything the bot was given. Anthropic's guide on reducing hallucinations describes it as the model producing text that is factually incorrect or inconsistent with the context it was given. The word is a nickname. The bot is not imagining anything. It is producing the most plausible-sounding text, and sometimes that text is wrong.
Why it happens
In a paper published in September 2025, researchers at OpenAI and Georgia Tech argue that models produce confident false statements partly because training and testing reward a confident guess more than an honest admission of doubt. If a test gives nothing for 'I don't know', a model that always answers scores higher than one that admits uncertainty. So answering is the default, even with nothing to base the answer on.
Three examples a founder will recognize
- An invented policy: a customer bot says refunds can be requested after the purchase when the real policy says otherwise. A tribunal held Air Canada responsible for exactly this kind of answer in 2024.
- An invented source: a lawyer's brief cites court cases that do not exist. Two lawyers were fined 5,000 dollars in 2023 after filing cases ChatGPT had made up.
- An invented number: you ask for last quarter's revenue and get a plausible figure from nowhere. This is why the honesty test asks exactly that question.
How it differs from a typo or an old answer
A typo is a slip in writing. An old answer was true once and nobody updated it. A hallucination is different: nothing true ever stood behind it. That matters because the fixes differ. An old answer needs a review date and an owner. A hallucination needs a bot that can only answer from approved sources and says so when it has none.
What to do about it
- Limit the bot to answers a person approved.
- Show the source and the date under every answer.
- Give it a fixed line for what it does not know, and send each unknown question to a person.
- Test it with questions that have no answer, and test again after every change.
- Keep a person responsible for anything that costs money or commits the company.
Hallucinations can be reduced, not removed
No. Anthropic says in its own guide that these techniques significantly reduce hallucinations but do not eliminate them, and that critical information should still be validated. You can check how your own bot behaves in a few minutes with the free honesty test at askmystartup.com/honesty-test. We ran it once on a made-up bakery workspace: our clone said it did not know in 36 of 36 tries and made nothing up.
Related guides
Common questions
No. The bot does not intend to deceive. It produces plausible text, and sometimes that text is false.
Researchers argue that models are trained and scored in ways that reward a confident guess over an honest 'I don't know', so producing an answer is the default.
Not from its tone, because wrong answers sound as sure as right ones. Check the source it points to, ask the same question twice, and test with questions that have no answer.
Better models can help, but the guidance we read says to keep validating anything critical. Treat any reduction as a reduction, not a cure.
Sources
- Anthropic: Reduce hallucinations (Claude documentation): platform.claude.com
- Kalai, Nachum, Vempala and Zhang: Why Language Models Hallucinate (arXiv, September 2025): arxiv.org
- Manatt: AI Gone Wild, Airline Has to Honor a Refund Policy Its Chatbot Fabricated: manatt.com
- LawNext: Court imposes sanctions on lawyers who filed bogus cases after relying on ChatGPT (June 2023): lawnext.com
Each page was opened and read on October 20, 2026. Our own numbers come from the honesty test page.