<b>What's the cleanest way to measure ad incrementality — and why do most teams get the control group wrong?</

<b>What's the cleanest way to measure ad incrementality — and why do most teams get the control group wrong?</b>

The question is really about counterfactual quality. A bad control silently destroys an otherwise rigorous test.

<b>The flawed default</b>

Many teams compare exposed users to unexposed users. This is hopelessly confounded: the people the algorithm <i>chose</i> to show ads to differ systematically from those it skipped. Exposed users were targeted precisely because they were likely to convert. You measure selection, not effect.

<b>Two designs that fix it</b>

— <i>PSA / placebo tests:</i> the control group is shown an unrelated charity ad in the slot your ad would have occupied. Now both groups passed the same auction and targeting filter; the only difference is creative.
— <i>Ghost ads:</i> the platform logs who <i>would have</i> won the auction for your ad in the control group without serving it. This is cheaper than PSAs and removes the cost of buying placebo impressions, while preserving the same selection mechanism.

<b>The nuance</b>

Ghost ads require platform cooperation and are only as trustworthy as the platform running them — the same entity grading its own homework. PSAs cost real money but are auditable by you.

<b>What to actually do</b>

— Insist the control be defined by the <i>auction</i>, not by exposure. "Would-have-been-shown" is the only valid counterfactual.
— Power the test before running it; underpowered lift studies produce confidently wrong zeros.

Bottom line for practitioners: incrementality is won or lost in the control group. If your unexposed group wasn't selected by the same machinery as your exposed group, you're measuring the algorithm, not your ads.