# Check AI output for errors and hallucinations

A verification checklist to catch fabricated facts, faulty logic, and unsupported claims before you rely on AI output.

- Topic: Working with AI
- Framework: RCTFVV
- Level: Advanced
- Tags: Analyze, Code, Fill-in Template
- Source: https://www.thinkwithprompt.com/prompts/check-ai-output-for-errors-and-hallucinations

## Prompt

```text
Role: Act as a data scientist responsible for validating AI-assisted analysis before use.

Context: The AI-generated output to validate (the analysis, numbers, code, or conclusions): [AI_OUTPUT]. The question it was meant to answer: [QUESTION]. Source data and definitions it should have used: [DATA_DEFINITIONS]. What decision depends on it and the stakes: [DECISION_STAKES]. Tools available to check: [TOOLS].

Goal: Catch incorrect calculations, unsupported assumptions, misinterpreted metrics, wrong filters and hallucinated conclusions before the result informs a decision.

Task: Produce: (1) a validation checklist covering - question fidelity (does it answer what was asked), definition and filter correctness, calculation correctness (recompute a sample), data lineage (are the figures traceable to the source), assumption validity, internal consistency (do the parts agree), sensitivity (does a small input change swing the conclusion implausibly), and hallucination check (are all cited facts/figures real); (2) the checklist applied to this output - pass/fail/uncertain per item with the evidence; (3) the issues found ranked by how much they threaten the conclusion; (4) a verdict - safe to use / use with caveats / do not use - with the reasoning and what to fix.

Constraints: Actually recompute at least one key figure and trace at least one number to the source - do not accept the AI's own claim that it checked. Treat any figure that cannot be traced to the source as unverified. State confidence for every checklist item.

Examples: Checklist item: 'Calculation correctness - recomputed [metric] from the source: AI said [X], recomputed [Y]. Result: FAIL, off by [diff] due to [cause].'

Format: Sections: Validation checklist | Checklist applied (table: item | result | evidence) | Issues ranked by risk | Verdict & fixes. ~600 words.

Verification: Be explicit about which checks you could complete with the tools/data available and which remain open; an open check is not a pass.

Validation: Given the decision stakes, is the verdict appropriately conservative? If any check is uncertain and the stakes are high, the verdict cannot be 'safe to use' - adjust.
```

## What to fill in

- AI_OUTPUT
- QUESTION
- DATA_DEFINITIONS
- DECISION_STAKES
- TOOLS
- metric
- diff
- cause

Free to copy and use. From ThinkWithPrompt (https://www.thinkwithprompt.com/), a library of 2,663 curated AI prompts.