# Diagnose why a model is performing worse in production

A step-by-step diagnosis separating data pipeline bugs, input drift, concept drift, and label issues, with a recovery plan.

- Topic: Data & Analysis
- Framework: Few-Shot
- Level: Standard
- Tags: Analyze, Code, Fill-in Template
- Source: https://www.thinkwithprompt.com/prompts/diagnose-why-a-model-is-performing-worse-in-production

## Prompt

```text
Help me diagnose why my model is performing worse in production than it did in testing or than it used to.

**Model and use:** [what it predicts, how outputs are used]
**What I'm seeing:** [e.g. "precision dropped from 0.62 to 0.41 over 6 weeks", "complaints from users"]
**Monitoring data available:** [PASTE input feature stats over time, prediction distribution, actual outcomes, error by segment]
**Recent changes:** [code releases, upstream data changes, product or policy changes, market events, retraining]

Work through, in order:
1. **Pipeline and serving bugs:** training-serving skew (features computed differently in production), missing or default-filled features, schema or unit changes, stale feature store, wrong model version deployed.
2. **Label issues:** are outcomes delayed, defined differently, or incomplete for recent periods, making performance look worse than it is?
3. **Data drift:** which input features changed distribution, and in which segments? Suggest measures (e.g. population stability index) and thresholds.
4. **Concept drift:** has the relationship between inputs and outcome changed (new customer behavior, pricing change, seasonality not seen in training)?
5. **Feedback loops:** is the model's own use changing the data it sees?

Return:
- A diagnostic checklist with the query or check for each step.
- Ranked hypotheses based on my evidence.
- Short-term mitigation options (rollback, threshold adjustment, fallback rules) and longer-term fixes (retraining window, new features, monitoring).
- Monitoring alerts I should add so this is caught earlier next time.
```

## What to fill in

- what it predicts, how outputs are used

Free to copy and use. From ThinkWithPrompt (https://www.thinkwithprompt.com/), a library of 2,663 curated AI prompts.