# Diagnose why my AI chatbot gives off-topic or irrelevant answers

Pinpoints whether the problem is the prompt, retrieval, context, or settings, and gives targeted fixes plus test cases to confirm them.

- Topic: Working with AI
- Framework: RISEN
- Level: Standard
- Tags: Communication, Analyze, Fill-in Template
- Source: https://www.thinkwithprompt.com/prompts/diagnose-why-my-ai-chatbot-gives-off-topic-or-irrelevant-answers

## Prompt

```text
Help me figure out why my chatbot gives off-topic or unrelated answers, and how to fix it.

**What the bot is for:** [PURPOSE and users]
**How it's built:** [model, platform or framework, whether it uses retrieval/RAG over documents, tools, conversation memory]
**System prompt:** [PASTE]
**Settings:** [temperature, max tokens, context length, retrieval top-k - if known]
**Bad examples:** [PASTE 3-5 user messages with the bot's replies, and what a good reply would have been]

Work through the likely causes in order and tell me which ones my examples point to:
1. **Instructions** - vague scope, conflicting rules, no instruction for what to do when a question is out of scope or the answer isn't known.
2. **Retrieval** - wrong or irrelevant chunks returned, chunks too large or small, missing metadata filters, no relevance threshold, the answer not in the knowledge base at all.
3. **Context handling** - conversation history truncated or bloated, earlier turns confusing the current one, user message not reaching the model intact.
4. **Settings** - temperature too high, output cut off.
5. **Input issues** - ambiguous user questions the bot should clarify instead of guessing.

Return:
- The most likely root causes, each tied to evidence from my examples.
- Specific fixes (rewritten system prompt sections, retrieval changes, settings), most impactful first.
- A test set of 10 user messages, including edge cases and out-of-scope questions, with the expected behavior for each, so I can check the fix worked.
```

## What to fill in

- PURPOSE and users
- PASTE

Free to copy and use. From ThinkWithPrompt (https://www.thinkwithprompt.com/), a library of 2,663 curated AI prompts.