How to stop a chatbot from making up answers
How to stop a chatbot from making things up: rank topics by risk, answer from authorized sources, limit what it can assert and review replies every week.
In a nutshell
There's no absolute guarantee that an AI will never be wrong. But you can greatly reduce the risk by limiting sources, authority and scope, and by having it acknowledge when it doesn't have the information instead of making it up.
Identify where a mistake hurts
Not every answer carries the same risk. Getting opening hours wrong is annoying; inventing a price, availability or a contractual condition can create a serious commitment.
Rank topics by impact and reserve the most sensitive ones for verifiable sources or human review.
- Low: general guidance.
- Medium: commercial policies.
- High: payments, availability or professional decisions.
Use authorized sources
The assistant should answer from approved content and keep it apart from general knowledge. If two documents contradict each other, the fix is to correct the source, not to ask the AI to choose.
Sources that change need frequent updates or an integration that looks up the data in real time.
- One current version per topic.
- Date and owner.
- No unnecessary internal data.
- Search and answer tests.
Use AI with your own business information.
Add products, policies and FAQs, and define what it can answer and when it hands off to your team.
Limit its authority
Spell out what it may state and what it must confirm. When it finds no backing, the right answer may be to say so and offer that the team continues.
Avoiding an answer is better than creating false certainty.
- Don't estimate stock.
- Don't create discounts.
- Don't confirm appointments without a calendar.
- Don't give regulated advice.
Ongoing monitoring
Review low-confidence replies, reopened conversations and team corrections. Turn every repeated finding into an improvement to the source or to the handoff rule.
Volga records the conversation's path and lets you pause the AI. Human supervision remains part of the system, especially in high-impact processes.
- Weekly sample.
- Adverse cases.
- Alerts for sensitive topics.
- Clear escalation path.
How to do it in Volga
In Volga, under Settings → “AI mode”, you choose how far the assistant goes: with “AI enabled” it replies using the information you loaded in Knowledge; with “Greeting only” it greets with the text you write and leaves the conversation to your team; with “AI disabled” it doesn't reply and the inquiry waits for your team. The change applies to messages that arrive after you save. “AI disabled” doesn't stop the welcome menu or the after-hours notice, if you have them.
If you'd rather a person reviews what the AI replies, turn on learner mode (Intelligence → “Learner mode”). The AI drafts the reply as a proposal in the Inbox, says which source it came from, and that reply doesn't reach the customer until someone on your team sends it: “Use” sends it as is, “Edit” lets you complete it before sending, and “Discard” asks why and sends nothing. Volga records what happened with each proposal, topic by topic, and when a topic is right 90% of the time for two weeks, with at least 30 proposals, it asks whether you want it to reply on its own.
Not everything becomes a proposal: topics you approved, conversations handled by Smart Intake, the “Greeting only” greeting and Instagram comments go out without waiting for anyone, and if you set your business hours, outside them the AI answers what it knows on its own, unless you choose “Wait for an agent” under “Outside business hours”. The learner mode guide has the details.
Under each assistant reply, “Sources” shows which information it came from; administrators and agents can see it. In the video, a customer asks about a product that isn't on the list and the assistant says it doesn't have that information, without making up a price. In Knowledge, “Analytics” shows how many times each source was used and when it was last used (administrators only). And if a reply went out with wrong information, you open the source, fix it and save: the source shows “Indexing” and, once it's “Ready”, replies use the new information (until then, the assistant uses the previous version).
To stop it in one chat, the conversation header has “Pause assistant”: the AI stops replying in that conversation, and “Resume assistant” gives it back. If someone on the team writes, it pauses by itself in that chat. In “AI mode”, “Disable AI in all” pauses it in open conversations; new inquiries still get the AI, so to stop it completely you choose “AI disabled”. When you pause or resume it in one conversation, the audit log, visible to administrators, keeps who did it, with the day and time; actions on all conversations at once aren't recorded there. Pausing the AI is available from the Starter plan.
If the customer asks to talk to a person and you have a team, the assistant hands the conversation over without any setup: the customer receives the text you wrote in “Message when handing off” (if you leave it empty, Volga uses its own), your team gets the “A customer wants to talk to a person” notification and the conversation shows up under “Waiting for a person”. With learner mode on, that message to the customer follows the rules above: where the AI proposes, it doesn't go out on its own, but your team still gets the notification. Until someone takes it, the assistant keeps answering what it knows; with “Take”, the conversation becomes yours and the AI pauses in that chat.
- Three modes: “AI enabled”, “Greeting only” and “AI disabled”.
- Learner mode: the AI proposes and your team uses, edits or discards.
- “Sources” under each reply and Knowledge “Analytics”.
- Pause in one conversation (recorded in the audit log) or in all of them.
- The customer who asks for a person: your message, the team notification and “Take”.
Frequently asked questions
Questions about this guide
Can you completely eliminate the risk of a wrong answer?
No. It shrinks with controlled sources, limits, validations and human handoff, but important processes always need supervision proportional to the risk.
Do more documents always improve the answer?
No. Many repeated or contradictory documents make search worse. What matters more is that the source is clear, current and authorized.