Measure and improve how well my customer chatbot performs
A metrics framework, a transcript review method, and a prioritized fix list for a support, sales, or e-commerce chatbot already in use.
The prompt
Help me evaluate and improve a chatbot that's already live.
What the chatbot does: [PURPOSE - e.g. "answers order and returns questions for an online store"]
Where it runs: [website / app / WhatsApp / Instagram / Messenger]
Languages it handles: [LIST]
What I can measure today: [e.g. "conversation logs, CSAT thumbs, handoff count"]
Main complaints or problems: [e.g. "loops on refund questions", "wrong answers in Spanish"]
Sample conversations: [PASTE 5-20 transcripts with personal data removed]
Give me:
1. A metrics set with definitions and how to calculate each: containment or resolution rate, escalation rate, fallback ("I didn't understand") rate, repeat contact within 7 days, CSAT, conversion or deflection value, and average turns to resolution. Say which matter most for my goal.
2. A review of my sample conversations: for each, whether the goal was met, where it went wrong (misunderstood intent, wrong or outdated answer, missing handoff, tone, language issue), and the fix.
3. A failure-pattern table: Pattern | Example | Frequency in the sample | Fix (content, flow, intent training, handoff rule).
4. Channel-specific issues for where it runs (message length, buttons, media, response time expectations).
5. If multilingual: checks for translation quality, mixed-language messages, local formats (dates, currency), and culturally off tone.
6. Privacy and safety checks: what personal data it collects, whether it asks for anything it shouldn't, and when it must hand off to a human.
7. A 30-day improvement plan, with the top 5 fixes ranked by impact and effort.
Base findings only on the transcripts and data I share. Don't invent benchmark rates; mark them [ADD BENCHMARK] if I need one.
Highlighted [brackets] are blanks to replace with your own details.
Fill in the blanks and use this promptMore Customer Support prompts
What to fill in
- website / app / WhatsApp / Instagram / Messenger
- LIST
- e.g. "conversation logs, CSAT thumbs, handoff count"
- e.g. "loops on refund questions", "wrong answers in Spanish"
- PASTE 5-20 transcripts with personal data removed
- ADD BENCHMARK
How this prompt is built
This prompt follows the Few-Shot structure (Few-shot prompting). A few-shot prompt includes one or more worked examples of the input and the output you want, so the model can copy the pattern instead of guessing it.
Related prompts
- Handle a warranty callback and decide what is coveredA fair way to assess a customer's warranty claim, decide responsibility, and reply clearly, whether the fix is covered or not.
- Explain a financial aid offer to a student and familyBreaks down a college aid offer into net cost, grants vs. loans, and next steps, with a clear reply to the family.
- Respond to a negative online review without breaching client confidentialityA short public reply to a bad Google or Avvo review that protects client confidences, plus guidance on whether to reply at all.
- Set up a simple log to track customer feedback and bug reportsA ready-to-use spreadsheet structure, tagging scheme, and weekly review routine for teams without a ticketing system.