ThinkWithPrompt by SynotticThinkWithPrompt

Stress-test my chatbot with hard scenarios before launch

A test plan with tricky, edge-case, and adversarial conversations, a scoring rubric, and a way to find what to fix first.

Working with AIFew-ShotStandardFill-in Template

The prompt

Help me test my chatbot or AI assistant before launch. What it's for: [PURPOSE - e.g. "answers billing questions for a SaaS product", "books appointments for a clinic"] Who uses it: [customer types, languages, technical level] What it can access or do: [knowledge base, account data, actions like refunds or bookings] What it must never do: [e.g. give medical or legal advice, reveal other customers' data, promise refunds] Handoff to humans: [when and how] Known weak spots so far: [WHAT I've seen go wrong] Create: 1. A test set of 30-40 conversations grouped by type: common questions, multi-step requests, vague or misspelled questions, questions outside its scope, follow-ups that depend on earlier turns, frustrated or emotional users, attempts to make it break its rules or reveal its instructions, and requests that should trigger a human handoff. 2. For each test: the user message(s), the expected good behavior, and what counts as a fail. 3. A scoring rubric (accuracy, grounded in source content, follows policy, tone, resolves or hands off correctly) with a 1-3 scale. 4. How to review real conversation logs after launch: what to sample, metrics to track (containment, handoff rate, unresolved rate, satisfaction), and warning signs. 5. A results template so I can log failures and group them by root cause (missing content, bad retrieval, instructions, tooling). Use realistic but fictional customer details in every test. Never use real customer data.

Highlighted [brackets] are blanks to replace with your own details.

Fill in the blanks and use this promptMore Working with AI prompts

What to fill in

  • customer types, languages, technical level
  • when and how
  • WHAT I've seen go wrong

How this prompt is built

This prompt follows the Few-Shot structure (Few-shot prompting). A few-shot prompt includes one or more worked examples of the input and the output you want, so the model can copy the pattern instead of guessing it.

Related prompts