# Design an incrementality or holdout test for a channel

A test design that measures whether a channel actually creates extra sales, not just credit for sales that would happen anyway.

- Topic: Data & Analysis
- Framework: Few-Shot
- Level: Standard
- Tags: Fill-in Template
- Source: https://www.thinkwithprompt.com/prompts/design-an-incrementality-or-holdout-test-for-a-channel

## Prompt

```text
Help me design an incrementality test to find out whether a marketing channel truly drives extra results.

**Channel or tactic to test:** [e.g. retargeting, branded search, Meta prospecting, direct mail]
**Why I'm questioning it:** [e.g. "platform reports a 9x ROAS, which seems too good"]
**Spend and volume:** [monthly spend, conversions, average order or deal value]
**Geography and audience:** [where we sell, audience sizes]
**Tools available:** [platform conversion lift studies, geo controls, CRM data]
**How long I can run it and what I can risk:** [e.g. "4 weeks, can pause 25% of regions"]

Design:
1. Pick the method that fits: platform lift study, audience holdout, geo-based test (matched markets, switch on/off), or time-based on/off. Explain the trade-offs for my case.
2. Define the treatment and control precisely, how to match them, and the primary metric (measured in my own system, not the ad platform).
3. Estimate whether my volume can detect a meaningful lift in the time available, and what to change if not (longer, bigger holdout, bigger spend change).
4. List contamination risks (other channels shifting, promotions, seasonality) and how to control them.
5. Show how to calculate incremental conversions, incremental ROAS or CPA, and the confidence range.
6. Decision rules agreed before launch: what result means scale, keep, or cut.

Return a one-page test plan and a results template. Remind me to verify sample-size math with an analyst if stakes are high.
```

## What to fill in

- e.g. "platform reports a 9x ROAS, which seems too good"
- monthly spend, conversions, average order or deal value
- where we sell, audience sizes
- platform conversion lift studies, geo controls, CRM data
- e.g. "4 weeks, can pause 25% of regions"

Free to copy and use. From ThinkWithPrompt (https://www.thinkwithprompt.com/), a library of 2,663 curated AI prompts.