Start free
A support agent chat grounded in a company's own documentation, with an uncertain question handed to a human

How to Stop Your AI Support Agent From Making Things Up

Published on byAntoniAntoni

Ask a founder what scares them about putting AI in front of customers and the answer is almost always the same: "What if it makes something up?" A confident, polite, completely invented refund policy is easy to picture. The good news is that hallucinations are not random. They have causes, the causes are mostly on your side of the fence, and each one has a fix. This is the playbook.

Key Takeaways


What a hallucination actually is

A language model does not look things up and report what it found. It produces the most likely next words given what is in front of it. With good material, the most likely words are usually the correct ones. When the material is missing, contradictory or vague, the words still come out; they are just no longer anchored to anything true.

That is a hallucination: a fluent, confident answer that no source supports. In support it looks like a plan limit that does not exist, a feature you never built, a "30-day" return window when yours is 14, or a shipping date nobody committed to. None of these are nonsense. They are all things a company like yours might say, which is what makes them dangerous: the customer has no reason to doubt them, and the model has no built-in way to know it is guessing.

Why it happens, in plain terms

1. Gaps in knowledge

The most common cause by far. A customer asks about something your content does not cover, and the agent has to choose between admitting it and improvising. Left to its own devices, improvising wins. If your docs explain data export but never say whether archived records are included, "yes, archived records are included" is exactly the sentence that gets generated.

2. Ambiguous questions

"Can I change my plan?" could mean upgrade, downgrade or switch billing period. If the agent picks one interpretation and runs with it, the answer may be perfectly correct for a question the customer did not ask. From the customer's side that is indistinguishable from being wrong.

3. Stale content

You changed your pricing in March. The pricing page is current, but the brochure PDF you uploaded last year and the launch blog post still show the old numbers. Now the agent has three sources, two of them wrong, and whichever it retrieves is the one it believes. Old content does not look old to a model. It just looks like content.

4. Over-eager helpfulness

Models are tuned to be useful, and "I don't know" feels unhelpful. Without an explicit instruction that not knowing is acceptable, the agent leans toward producing an answer rather than the honest non-answer. This is why hallucinations cluster around high-stakes questions, where a customer presses for a definitive yes or no.

The Bot never hallucinated because it never understood anything; it matched a keyword and read a script. An agent reasons over your knowledge, which is far more useful, and that reasoning has to be grounded or it drifts. Grounding is the whole game.

The prevention playbook

Step 1: Ground every answer in your own content

The foundation is simple: the agent answers from your website, docs, FAQs and files, and from nothing else. Not from its general training, not from what "companies like yours usually do". If a customer asks something your content does not cover, the right output is "I could not find that", not a best guess.

In FutureBase that is the default: the agent answers only from your knowledge, and the Ask your agent tester shows which items it used for any answer. Whatever tool you use, make sure you can see the sources behind an answer. If you cannot trace it back to a document, you cannot tell a good answer from a lucky one.

Grounding also exposes every topic your docs skip. Each "I could not find that" is a content gap with a customer's name on it.

Step 2: Use exact answers for anything that must be exact

Retrieval plus generation is great for explaining how a feature works. It is the wrong tool for a refund window. For answers that must be precise and identical every time, write them once as verbatim question-and-answer pairs that take priority over general content: pricing and plan limits, refund and cancellation terms, shipping times, warranty, data retention, anything with a number or a deadline in it. Write them the way a customer would ask, one intent per entry, with the complete answer so the agent has nothing to add. "How do I request a refund?" should map to the full process, the time limit and the payment-method detail, not to "contact support".

Step 3: Keep content current, and kill the duplicates

Stale content is the cause nobody budgets time for. Three habits fix most of it:

For more on structuring content so the agent retrieves the right thing, see how to train AI on your documentation.

Step 4: Make "I don't know" an explicit, good outcome

This is the instruction most people forget. Tell the agent, in plain words, that not finding an answer is acceptable and what to do about it:

If the answer is not in the provided content, say that you could not find it and offer to connect the customer with the team. Do not guess.

Then make the fallback itself useful. A dead-end "I don't understand" is the Bot's signature move; an honest "I could not find that in our docs, let me get a person to help" is a resolution in progress. Write that fallback message deliberately and pair it with a real handoff.

The handoff is non-negotiable. When the agent cannot help, or the customer asks for a person, the conversation should land with your team with the full transcript. In FutureBase this is the Hand off to your team action, which waits in Live Desk and can notify you by email, Slack or Discord; see the actions guide. The honest path should be easy for the agent to take, so it never feels pressure to improvise.

Step 5: Scope the agent: what it must never promise

Hallucinations do the most damage when they create a commitment. Decide, in writing, the categories where the agent may inform but must never decide, and give it a safe behaviour for each.

Risky question typeWhy it is riskySafe behaviour
"Will I get a refund?"A yes is a promise you may not honourState the written policy, never predict the outcome, hand to a person
"When will my order / the feature ship?"Invented dates become broken promisesShare only published timelines; otherwise say you do not have a date and offer to follow up
Legal, tax, complianceWrong answers create liabilityPoint to the published terms, recommend a professional, do not interpret
Medical, safety, healthHarm to the customerDo not advise; escalate immediately
Account security, passwords, card numbersSocial engineering and data exposureNever ask for credentials or payment details; route to a verified process
Exceptions to policy ("can you make an exception?")Agent has no authorityAcknowledge, explain the standard policy, hand to a person who can decide
Angry or distressed customersNeeds empathy and judgementApologise sincerely, skip the troubleshooting, hand off with context

Put these into the agent's custom instructions as concrete rules. "Never confirm whether a specific refund will be approved; share the refund policy and offer to connect them with the team" is a rule the agent can follow. "Be careful about refunds" is not.

Step 6: Test with real past questions, not invented ones

Before launch, and after every meaningful content change, run a test sheet with questions pulled from your actual inbox, chat history or ticket queue. Real customers phrase things badly, combine two questions in one, and ask about edge cases you would never think to test. Those are the questions that trigger hallucinations.

For each question, check three things:

  1. Is the answer correct? Compare it to the source, not to your memory.
  2. Is it grounded? A correct answer drawn from the wrong page is a hallucination waiting to happen on a slightly different question.
  3. Does it stop at the right point? On a risky question, did the agent inform and hand off, or did it decide?

Fifty real questions is a good first pass. Fix every wrong answer at the content level before touching the instructions. Content fixes are durable; instruction tweaks mostly paper over a missing fact.

Step 7: Review weak answers every week

After launch, the hallucinations you have not caught are hiding in conversations that were never escalated. Nobody complained; they quietly got a wrong answer and left. So the review cannot be limited to failures.

Sample conversations the agent "resolved" each week and run the same three checks. Look for thumbs-down ratings, repeat contacts on the same topic, and answers that contain a number or a date. Group what you find by topic: if three customers asked about archived exports and the agent fumbled all three, that is one FAQ away from fixed.

Step 8: Tighten instructions one rule at a time

Resist the urge to write a wall of rules up front. Over-restricted agents become unhelpful in a different way: they refuse reasonable questions, hedge everything, and send customers to a human for things the docs plainly answer. That is the Bot with better grammar.

Start with the defaults plus your scoping rules from Step 5, then add a rule only when a review surfaces a specific, repeated problem. Each rule names the situation and the behaviour: "When a customer asks about delivery times, give the published 3 to 5 business day range and do not estimate a specific date." Test it against the question that prompted it and a couple of neighbouring ones, and note why each rule exists so you can remove it once the content is fixed.

The whole system in one exchange

A customer writes: "I ordered on Friday and need it by Wednesday, will it arrive?"

An ungrounded agent says "Yes, your order will arrive by Wednesday." Nobody knows that. If it does not arrive, the customer has a written promise and a reason to be angry.

A grounded, scoped agent says: "Standard shipping is 3 to 5 business days from dispatch, so an order placed Friday usually arrives between Wednesday and Friday. I cannot guarantee a specific day, but share your order number and I will get someone from the team to check the tracking." Real policy, no invented date, and a path to a person who can look. That is the agent you are building: useful precisely because it knows the edge of what it knows.

Frequently asked questions

Can you completely eliminate AI support hallucinations?

No. Any system that generates language can produce a wrong sentence. What you can do is make hallucinations rare and harmless: ground answers in your own content, keep that content current, give the agent an explicit way to say it does not know, hand risky questions to a person, and review weak answers regularly. The goal is an agent that is wrong less often than a tired human and never wrong about the things that cost you money.

Why does my AI agent give different answers to the same question?

Usually because your content gives it more than one answer. Two pages that disagree, a page and a PDF with different numbers, or a stale FAQ next to a fresh policy will produce inconsistent replies. Find the contradiction, fix it at the source, and put the exact answer in an FAQ so there is one place the agent can pull it from.

Should my AI agent ever say it does not know?

Yes, and it should say so plainly. A clear answer like I could not find that in our docs, let me get a person to help, is a good outcome. It is honest, it keeps the customer moving, and it tells you exactly what content is missing. A confident guess that turns out wrong is far more expensive than an honest handoff.


Want an agent that answers from your docs and hands off when it should? Start free with FutureBase: import your site, add FAQs for the answers that must be exact, test with real questions, and turn on handoff so no customer hits a dead end. See how teams run automated customer support and knowledge base agents without the guessing.

Retire the Bot. Give your customers an agent.

Deploy in five minutes. No credit card required.

Trained on your website in minutes
Knowledge base & file upload
Help desk with full-context hand-off
MCP server access
GDPR compliant