Design an A/B test
Hypothesis, variants, metrics, sample size, and duration for a valid test.
The prompt
Help me design an A/B test.
What I'm testing: [e.g. landing page headline, email subject line, pricing page, onboarding step]
Current version and its performance: [baseline metric, e.g. "3.2% conversion"]
Change I want to test: [VARIANT]
Traffic/volume available: [visitors or sends per week]
Business goal: [WHAT SUCCESS MEANS]
Give me:
1. A clear hypothesis: "If we [change], then [metric] will [direction] because [reason]."
2. Primary metric and 1–2 guardrail metrics (things that must not get worse).
3. Sample size and test duration needed to detect a realistic effect — show the assumptions (baseline, minimum detectable effect, 95% confidence, 80% power).
4. Test setup: randomization, what to keep constant, how long to run (full weeks), and when NOT to stop early.
5. A decision rule written before the test starts.
6. Common pitfalls for this kind of test.
If my traffic is too low for a reliable test, say so and suggest alternatives.
Highlighted [brackets] are blanks to replace with your own details.
Fill in the blanks and use this promptMore Data & Analysis prompts
What to fill in
- baseline metric, e.g. "3.2% conversion"
- VARIANT
- visitors or sends per week
- WHAT SUCCESS MEANS
- change
- metric
- direction
- reason
How this prompt is built
This prompt follows the Few-Shot structure (Few-shot prompting). A few-shot prompt includes one or more worked examples of the input and the output you want, so the model can copy the pattern instead of guessing it.
Related prompts
- Break down my results into the business drivers behind themBuilds a driver tree from your results, sizes how much each driver contributed to the change, and names which ones to act on.
- Identify the leading and lagging indicators for my business or teamSorts your metrics into leading and lagging indicators, links each lead to the outcome it predicts, and fills gaps in early-warning measures.
- Find the 20% of activities driving 80% of my resultsRuns a Pareto analysis to show the few activities, customers, or products producing most of your results, and what to cut or double down on.
- Pick the three insights and five numbers my executives needRanks your findings down to the single most important insight, the top three, the five numbers to watch, and the biggest surprise.