# Measure the impact of a change when an A/B test isn't possible

Chooses and plans a quasi-experimental method like difference-in-differences or synthetic control, with assumptions and checks.

- Topic: Data & Analysis
- Framework: Few-Shot
- Level: Standard
- Tags: Explain & Teach, Fill-in Template, Step-by-step
- Source: https://www.thinkwithprompt.com/prompts/measure-the-impact-of-a-change-when-an-a-b-test-isn-t-possible

## Prompt

```text
Help me measure the causal impact of a change that was not run as a randomized A/B test.

**What changed:** [e.g. "new pricing launched in 3 regions", "feature released to all users on May 1"]
**How it was rolled out:** [all at once / staggered by region or date / opt-in / eligibility rule or threshold]
**Outcome metric(s):** [definitions]
**Data available:** [time range before and after, units (users, stores, regions), possible comparison groups, other events in the period]

Steps:
1. Recommend the most credible method for my rollout, and explain why, choosing among: difference-in-differences, synthetic control, regression discontinuity, interrupted time series, matching or propensity scores, and staggered-rollout designs.
2. State the key assumption the method relies on (e.g. parallel trends) in plain English, and how to test it with my data.
3. Define the treatment and comparison groups, the time windows, and what to exclude.
4. List confounders and other events that could explain the change, and how to handle each.
5. Describe robustness checks: placebo tests, alternative comparison groups, different windows.
6. Explain how to express uncertainty (confidence intervals) and what result would be "inconclusive".

Return:
- The recommended method and a one-paragraph rationale.
- A step-by-step analysis plan with the data needed for each step.
- Example code outline in [Python / R / SQL].
- A template for reporting the result, with language that is honest about how strong the causal claim is.
```

## What to fill in

- definitions
- Python / R / SQL

Free to copy and use. From ThinkWithPrompt (https://www.thinkwithprompt.com/), a library of 2,663 curated AI prompts.