Stress-test my chatbot with hard scenarios before launch
A test plan with tricky, edge-case, and adversarial conversations, a scoring rubric, and a way to find what to fix first.
The prompt
Help me test my chatbot or AI assistant before launch.
What it's for: [PURPOSE - e.g. "answers billing questions for a SaaS product", "books appointments for a clinic"]
Who uses it: [customer types, languages, technical level]
What it can access or do: [knowledge base, account data, actions like refunds or bookings]
What it must never do: [e.g. give medical or legal advice, reveal other customers' data, promise refunds]
Handoff to humans: [when and how]
Known weak spots so far: [WHAT I've seen go wrong]
Create:
1. A test set of 30-40 conversations grouped by type: common questions, multi-step requests, vague or misspelled questions, questions outside its scope, follow-ups that depend on earlier turns, frustrated or emotional users, attempts to make it break its rules or reveal its instructions, and requests that should trigger a human handoff.
2. For each test: the user message(s), the expected good behavior, and what counts as a fail.
3. A scoring rubric (accuracy, grounded in source content, follows policy, tone, resolves or hands off correctly) with a 1-3 scale.
4. How to review real conversation logs after launch: what to sample, metrics to track (containment, handoff rate, unresolved rate, satisfaction), and warning signs.
5. A results template so I can log failures and group them by root cause (missing content, bad retrieval, instructions, tooling).
Use realistic but fictional customer details in every test. Never use real customer data.
Highlighted [brackets] are blanks to replace with your own details.
Fill in the blanks and use this promptMore Working with AI prompts
What to fill in
- customer types, languages, technical level
- when and how
- WHAT I've seen go wrong
How this prompt is built
This prompt follows the Few-Shot structure (Few-shot prompting). A few-shot prompt includes one or more worked examples of the input and the output you want, so the model can copy the pattern instead of guessing it.
Related prompts
- Build a reusable prompt templateCreates a fill-in-the-blanks template for a task you repeat often.
- Find where AI can help in my work or businessInterviews you about a process and identifies where AI genuinely helps — and where it doesn't.
- Build an AI adoption roadmap for my team or functionA phased plan for adopting AI in sales, marketing, HR, finance, legal, engineering, support, or operations — with use cases, guardrails, and metrics.
- Remove sensitive information before pasting into AIFinds and replaces personal, confidential, and secret data with safe placeholders.