# Design an A/B test

Hypothesis, variants, metrics, sample size, and duration for a valid test.

- Topic: Data & Analysis
- Framework: Few-Shot
- Level: Standard
- Tags: Fill-in Template
- Source: https://www.thinkwithprompt.com/prompts/design-an-a-b-test

## Prompt

```text
Help me design an A/B test.

**What I'm testing:** [e.g. landing page headline, email subject line, pricing page, onboarding step]
**Current version and its performance:** [baseline metric, e.g. "3.2% conversion"]
**Change I want to test:** [VARIANT]
**Traffic/volume available:** [visitors or sends per week]
**Business goal:** [WHAT SUCCESS MEANS]

Give me:
1. A clear hypothesis: "If we [change], then [metric] will [direction] because [reason]."
2. Primary metric and 1–2 guardrail metrics (things that must not get worse).
3. Sample size and test duration needed to detect a realistic effect — show the assumptions (baseline, minimum detectable effect, 95% confidence, 80% power).
4. Test setup: randomization, what to keep constant, how long to run (full weeks), and when NOT to stop early.
5. A decision rule written before the test starts.
6. Common pitfalls for this kind of test.

If my traffic is too low for a reliable test, say so and suggest alternatives.
```

## What to fill in

- baseline metric, e.g. "3.2% conversion"
- VARIANT
- visitors or sends per week
- WHAT SUCCESS MEANS
- change
- metric
- direction
- reason

Free to copy and use. From ThinkWithPrompt (https://www.thinkwithprompt.com/), a library of 2,663 curated AI prompts.