Why your AI chatbot gives wrong answers, and how to fix it

A chatbot that is right most of the time and confidently wrong the rest is worse than no chatbot, because customers cannot tell which answer they got. Owners often assume the model itself is the problem and start shopping for a new tool. The cause is usually closer to home: the documents it reads, the instructions it was given, and the fact that nobody tests it after launch.

Why does an AI chatbot make things up?

Language models are built to produce a plausible answer, and when they lack the right information they often guess rather than admit it. Researchers at OpenAI argued in a 2025 paper that common training and evaluation methods reward guessing over saying "I'm not sure", much as a multiple-choice exam rewards a guess over a blank. You cannot remove this tendency from the model, but you can control what it reads and what it is allowed to say.

What are the usual causes of wrong answers?

Most wrong answers trace back to one of five causes.

1. Stale or conflicting sources. The bot reads an old price list, last year's policy and the current one, and picks the wrong one.
2. Retrieval pulling the wrong passage. Bots that answer from your documents first search them for relevant text. If the search grabs a paragraph about a different product or location, the answer will be confidently about the wrong thing.
3. No instruction to say "I don't know". If the instructions never tell the bot to decline or hand off when the answer is not in its sources, it fills the gap with a guess.
4. Questions outside its scope. A bot built to answer questions about services gets asked about refunds, legal disputes or medical advice, and tries anyway.
5. A model or platform update. Vendors update models and settings. A bot that behaved well in spring can drift by autumn with nobody changing anything on your side.

How do you diagnose a chatbot in 30 minutes?

Pull 20 real conversations where the bot got something wrong, then label each one with one of the five causes above. Most owners find that one or two causes account for most of the errors. That tells you where to spend your time.

Example: a 25-person property management company in Halifax runs a website chatbot for rental enquiries. A prospective tenant is told that cats are allowed in a building where the condo board banned pets the year before. Reviewing 20 bad answers, the office manager finds that 11 came from an old policy PDF still in the bot's knowledge folder, 5 were questions about lease disputes the bot should never have answered, and 4 were retrieval mix-ups between two buildings with similar names. Cleaning the source folder and adding a handoff rule for lease questions would deal with 16 of the 20.

How do you fix each cause?

• Stale sources: keep one current version of each document, delete or archive the rest, and put a named person in charge of updating them when prices or policies change.
• Wrong passages: split long documents into clear sections with headings, label which location or product each one covers, and have the bot show the source it used so staff can spot mix-ups.
• Guessing: add an explicit rule: if the answer is not in the sources, say so and offer to connect the person with staff.
• Out-of-scope questions: list what the bot handles and what it hands off. Anything involving money, commitments, complaints, legal or health questions goes to a person.
• Vendor updates: re-run your test set whenever the vendor announces a change, and at least once a month.

For anything that commits your business to a price, refund, booking or exception, add a human approval step. It costs a little speed and saves a lot of trouble.

How do you test a chatbot properly?

Build a test set: 50 to 100 real questions from past conversations, each with the answer you expect. Include the awkward ones, such as questions it should decline, questions with no answer in your documents, and questions that mention two locations at once. Run the full set after every change to the bot, its documents or its instructions, and note which answers changed. A short spreadsheet works. What matters is that the same questions get asked every time, so you can tell whether a change helped or made things worse.

Are you liable for what your chatbot says?

In at least one Canadian case, yes. In Moffatt v. Air Canada, 2024 BCCRT 149, a passenger relied on the airline's website chatbot, which told him he could apply for a bereavement fare after travelling. Another page on the same website said otherwise. Air Canada argued the chatbot was responsible for its own actions. The BC Civil Resolution Tribunal called that "a remarkable submission" and found the airline liable for negligent misrepresentation, because the chatbot was part of its website. It ordered Air Canada to pay the fare difference plus interest and fees.

It was a small claims tribunal and the amount was small. Still, treat your chatbot's answers as statements your business is making.

How do you keep a chatbot accurate after launch?

Give one person ownership and a routine.

• A weekly review of flagged or low-rated conversations, 20 to 30 minutes.
• A simple way for staff to mark a wrong answer they see in a transcript.
• A monthly run of the full test set, plus one after any vendor update.
• A dated log of every change to the documents or instructions, so you can undo one that made things worse.

Should you repair your chatbot or rebuild it?

Repair it if most errors come from documents, instructions or scope, which is the usual case. Rebuild it if the platform cannot show which source it used, cannot be told to decline, cannot hand off to a person, or locks your content in a format you cannot export. Those are limits of the tool, and no amount of document cleaning fixes them.

What to do next

• Pull 20 recent wrong answers and label each by cause.
• Clean out old and duplicate documents from the bot's sources.
• Add a "say you don't know and hand off" rule.
• Write down what the bot must never answer on its own.
• Build a test set of at least 50 real questions and run it after every change.
• Name one person who owns the bot's accuracy.

If your bot answers enquiries, our guide to responding to enquiries faster covers where a human should stay in the loop. If you would rather have someone diagnose and repair it, that is what our AI workflow rescue and quality care service does.