Checklist for testing a chatbot before launch
Normal, ambiguous, sensitive and failure cases to validate replies, limits and handoffs before serving customers.
In a nutshell
A happy-path demo isn't enough. Testing should cover incomplete questions, information that doesn't exist, out-of-scope requests, channel errors and the handoff to the human team.
Build a representative set
Use real anonymized questions and add variants with typos, abbreviations and several pieces of data in one sentence. Cover the main intents and times of day.
Save the set so you can rerun it when a source or rule changes.
- Frequent questions.
- Informal language.
- Incomplete data.
- Change of intent.
Test the limits
Ask about information that doesn't exist, unauthorized discounts, sensitive topics and actions the integration can't confirm.
The expected reply should acknowledge the limit and offer a real way forward.
- Don't invent.
- Don't reveal internal data.
- Don't confirm fictitious actions.
- Hand off clearly.
Test the whole system
Check inbound and outbound messages, attachments, identity, assignment, AI pause and notifications. One good isolated reply doesn't prove the process works.
Include mobile and unstable network situations when they're part of daily use.
- Real test channel.
- Inbox and owners.
- Handoff and return.
- Status log.
Exit criteria
Define which failures block launch and who signs off on the result. Start with a limited scope and frequent monitoring.
Volga offers a playground to test the assistant and an inbox to review the behavior served. Validation should use the same configuration that will reach customers.
- Zero known critical errors.
- Current sources.
- Handoff verified.
- Owner for monitoring.
Frequently asked questions
Questions about this guide
How many questions should you test?
Enough to cover intents, variants and risks. Scenario coverage matters more than a fixed number.
Is testing in a simulator enough?
Not for sign-off. You also need to validate the channel, configuration and inbox that customers and the team will use.