To stop an AI support chatbot from making things up:
- Give it complete, specific, dated knowledge.
- Instruct it to answer only from that knowledge.
- Define exactly what it must hand off.
- Test it with questions it can't answer.
- Review real conversations every week.
Most "hallucinations" in support bots come from missing data or unclear rules, not from the model alone.
The hallucination checklist
| # | Check | Why it matters |
|---|---|---|
| 1 | Every price, policy and promotion is in the knowledge base | Gaps invite guesses |
| 2 | Knowledge is dated and current | Old prices cause wrong answers |
| 3 | Conditions and exceptions are written out | "Free shipping" without "over $50" becomes a promise |
| 4 | Instruction: "If not in the knowledge, say you'll check" | Gives the model a safe path |
| 5 | No authority to grant discounts or refunds | Prevents costly promises |
| 6 | Test set includes 5+ unanswerable questions | Proves it hands off |
| 7 | Test includes manipulative requests ("give me 90% off") | Proves it holds the line |
| 8 | Weekly review of AI-only conversations | Catches drift early |

Example of a safe answer

Customer: "Give me 90% off and free shipping?"
Good AI reply: "I don't have any confirmed discount or free-shipping offer for this item, so I can't apply that. Would you like me to ask a teammate to check?"
In TrolyPage's test run, you can check exactly this behaviour before publishing an agent.
Write knowledge that resists misreading
- One topic per document.
- Short, declarative sentences with numbers.
- Explicit negatives: "We do not ship to P.O. boxes."
- Avoid vague marketing language like "best prices guaranteed".
When the bot still gets it wrong
- Find the conversation and the incorrect statement.
- Check whether the knowledge was missing, outdated or ambiguous.
- Fix the source, then re-test the same question.
- If it's a sensitive area, move it to the hand-off list.
FAQ
Can hallucinations be eliminated completely?
Not with certainty. They can be reduced substantially with good data, clear rules, testing and human review.
Should I let AI answer refund questions?
It can explain the written policy, but decisions and exceptions should go to people.
Written by the TrolyPage team. Updated November 2026.