← All guides · Using AI well
How to tell when an AI answer is wrong
AI chatbots are wrong in a specific way: fluently, confidently, and in the same tone they use when they're right. You can't hear the difference, so you have to check for it. Here's how, in about a minute per answer.
Why AI sounds sure even when it isn't
A language model writes the most plausible next words given everything before them. Most of the time plausible and true overlap. When they don't, say a niche fact, a recent event, an exact number, or a quote, the model still produces something that sounds like an answer. There's no built-in alarm that goes off when it runs out of real knowledge. People call this "hallucination", but it's closer to filling a gap with a good guess and not mentioning that it guessed.
So the useful question isn't "is this AI smart?" It's "which kinds of claims in this answer are the ones it tends to guess?"
The claims most likely to be made up
- Exact numbers. Dates, prices, percentages, populations, version numbers. A model can get the shape of a fact right and the number wrong.
- Citations and quotes. Paper titles, authors, page numbers, and "as X once said" lines are easy to fabricate and hard to spot, because they look exactly like real ones.
- Anything recent. Every model has a training cutoff. Without a live web search it simply doesn't know what happened after that, and may describe an older state of the world as current.
- Small or local things. A minor company, a local law, a specific school's rules, a person who isn't famous. Less training data means more guessing.
- Code that calls libraries. Function names and options that sound right but don't exist, or existed three versions ago.
Broad explanations ("how does compound interest work?") are much safer than narrow specifics ("what's the interest rate at this bank?").
A one-minute check you can do on any answer
- Find the load-bearing claim. Which one sentence, if wrong, makes the rest useless? Check that one, not everything.
- Ask where it came from. "What's your source for that?" If it names a source, open it yourself. If the link is dead or the page doesn't say that, you have your answer.
- Ask it to argue the other side. "What would someone who disagrees say?" A shaky answer often falls apart when the model has to steelman the opposite.
- Rephrase and ask again. Ask the same question a different way, ideally in a new chat. Facts it actually knows tend to come back the same. Guesses tend to drift.
- Check one primary source. For anything that matters, one official page (the law itself, the manufacturer's spec sheet, the library's docs) beats any number of confident paragraphs.
Red flags in the wording
- Very specific details nobody asked for, like an exact year, a full name, or a precise statistic, dropped into an otherwise general answer.
- A citation that's oddly perfect: the exact title you'd hope existed, by an author with a generic name.
- The answer changes when you push back even slightly ("are you sure?"). A model that folds instantly was guessing.
- It agrees with a false premise in your question. If you ask "why did X happen?" and X never happened, a weak answer explains it anyway.
When you don't need to check
Not everything needs verifying. Brainstorming, rewording your own writing, explaining a concept you can sanity-check, or drafting something you'll edit anyway are low-risk. The more an answer will be used as fact by someone else, like in an essay, a medical or legal decision, money, or production code, the more checking it deserves.
How Sevoria helps with this
Sevoria is built to say "I don't know that one" rather than invent an answer, but no model gets that right every time, which is why the chat itself reminds you to verify important information. Two features make checking faster: turn on web search and replies cite the pages they used, so you can open the source instead of trusting the summary. And if a reply is wrong, use Report on that message; reports go to a real review queue. The Limitations page lists what Sevoria is known to get wrong.
FAQ
Do some AI models hallucinate less than others?
Yes, larger and newer models generally guess less, and models with web search guess less about recent facts. None of them reach zero, so the check above still applies.
If I tell the AI to "only say true things", does that work?
It helps a little, mostly by making it more willing to say it isn't sure. It doesn't give the model knowledge it doesn't have.
Can I trust an AI answer that includes a link?
Only after you open the link. A real-looking URL is not evidence; the page saying the same thing is.
Try it with web search on See Sevoria's limitations
Was this useful?
Thanks, noted.