Diagnose why my AI chatbot gives off-topic or irrelevant answers

Help me figure out why my chatbot gives off-topic or unrelated answers, and how to fix it. What the bot is for: [PURPOSE and users] How it's built: [model, platform or framework, whether it uses retrieval/RAG over documents, tools, conversation memory] System prompt: [PASTE] Settings: [temperature, max tokens, context length, retrieval top-k - if known] Bad examples: [PASTE 3-5 user messages with the bot's replies, and what a good reply would have been] Work through the likely causes in order and tell me which ones my examples point to: 1. Instructions - vague scope, conflicting rules, no instruction for what to do when a question is out of scope or the answer isn't known. 2. Retrieval - wrong or irrelevant chunks returned, chunks too large or small, missing metadata filters, no relevance threshold, the answer not in the knowledge base at all. 3. Context handling - conversation history truncated or bloated, earlier turns confusing the current one, user message not reaching the model intact. 4. Settings - temperature too high, output cut off. 5. Input issues - ambiguous user questions the bot should clarify instead of guessing. Return: - The most likely root causes, each tied to evidence from my examples. - Specific fixes (rewritten system prompt sections, retrieval changes, settings), most impactful first. - A test set of 10 user messages, including edge cases and out-of-scope questions, with the expected behavior for each, so I can check the fix worked.