# Measure and improve how well my customer chatbot performs

A metrics framework, a transcript review method, and a prioritized fix list for a support, sales, or e-commerce chatbot already in use.

- Topic: Customer Support
- Framework: Few-Shot
- Level: Advanced
- Tags: Customer Support, Analyze, Fill-in Template
- Source: https://www.thinkwithprompt.com/prompts/measure-and-improve-how-well-my-customer-chatbot-performs

## Prompt

```text
Help me evaluate and improve a chatbot that's already live.

**What the chatbot does:** [PURPOSE - e.g. "answers order and returns questions for an online store"]
**Where it runs:** [website / app / WhatsApp / Instagram / Messenger]
**Languages it handles:** [LIST]
**What I can measure today:** [e.g. "conversation logs, CSAT thumbs, handoff count"]
**Main complaints or problems:** [e.g. "loops on refund questions", "wrong answers in Spanish"]
**Sample conversations:** [PASTE 5-20 transcripts with personal data removed]

Give me:
1. A metrics set with definitions and how to calculate each: containment or resolution rate, escalation rate, fallback ("I didn't understand") rate, repeat contact within 7 days, CSAT, conversion or deflection value, and average turns to resolution. Say which matter most for my goal.
2. A review of my sample conversations: for each, whether the goal was met, where it went wrong (misunderstood intent, wrong or outdated answer, missing handoff, tone, language issue), and the fix.
3. A failure-pattern table: Pattern | Example | Frequency in the sample | Fix (content, flow, intent training, handoff rule).
4. Channel-specific issues for where it runs (message length, buttons, media, response time expectations).
5. If multilingual: checks for translation quality, mixed-language messages, local formats (dates, currency), and culturally off tone.
6. Privacy and safety checks: what personal data it collects, whether it asks for anything it shouldn't, and when it must hand off to a human.
7. A 30-day improvement plan, with the top 5 fixes ranked by impact and effort.

Base findings only on the transcripts and data I share. Don't invent benchmark rates; mark them [ADD BENCHMARK] if I need one.
```

## What to fill in

- website / app / WhatsApp / Instagram / Messenger
- LIST
- e.g. "conversation logs, CSAT thumbs, handoff count"
- e.g. "loops on refund questions", "wrong answers in Spanish"
- PASTE 5-20 transcripts with personal data removed
- ADD BENCHMARK

Free to copy and use. From ThinkWithPrompt (https://www.thinkwithprompt.com/), a library of 2,663 curated AI prompts.