Measure the impact of a change when an A/B test isn't possible
Chooses and plans a quasi-experimental method like difference-in-differences or synthetic control, with assumptions and checks.
The prompt
Help me measure the causal impact of a change that was not run as a randomized A/B test.
What changed: [e.g. "new pricing launched in 3 regions", "feature released to all users on May 1"]
How it was rolled out: [all at once / staggered by region or date / opt-in / eligibility rule or threshold]
Outcome metric(s): [definitions]
Data available: [time range before and after, units (users, stores, regions), possible comparison groups, other events in the period]
Steps:
1. Recommend the most credible method for my rollout, and explain why, choosing among: difference-in-differences, synthetic control, regression discontinuity, interrupted time series, matching or propensity scores, and staggered-rollout designs.
2. State the key assumption the method relies on (e.g. parallel trends) in plain English, and how to test it with my data.
3. Define the treatment and comparison groups, the time windows, and what to exclude.
4. List confounders and other events that could explain the change, and how to handle each.
5. Describe robustness checks: placebo tests, alternative comparison groups, different windows.
6. Explain how to express uncertainty (confidence intervals) and what result would be "inconclusive".
Return:
- The recommended method and a one-paragraph rationale.
- A step-by-step analysis plan with the data needed for each step.
- Example code outline in [Python / R / SQL].
- A template for reporting the result, with language that is honest about how strong the causal claim is.
Highlighted [brackets] are blanks to replace with your own details.
Fill in the blanks and use this promptMore Data & Analysis prompts
What to fill in
- definitions
- Python / R / SQL
How this prompt is built
This prompt follows the Few-Shot structure (Few-shot prompting). A few-shot prompt includes one or more worked examples of the input and the output you want, so the model can copy the pattern instead of guessing it.
Related prompts
- Explain a complicated Excel formula in plain EnglishBreaks down a long or nested Excel formula step by step, says what it actually returns, and points out hidden risks.
- Build a simple spreadsheet to track ticket sales for an eventStep-by-step setup for a ticket sales tracker with a data entry form, search, totals, and payment status for fundraisers and community events.
- Choose and interpret the right statistical testPicks the right method for your question and data and explains results in plain language.
- Break down my results into the business drivers behind themBuilds a driver tree from your results, sizes how much each driver contributed to the change, and names which ones to act on.