<b>How do you A/B test an ad you can't withhold?</b>

<b>How do you A/B test an ad you can't withhold?</b>

The question: classic incrementality testing means showing ads to a treatment group and nothing to a control. But on auction platforms you can't just 'not bid' cleanly — the control group still sees <i>some</i> ad, and the users who would have seen yours are exactly the ones you lose visibility into. That selection problem quietly poisons naive lift tests.

What the research says: this is why 'ghost ads' and 'public service announcement (PSA) controls' exist. In a ghost-ad design, the platform records which control-group users <i>would have</i> won your auction and served them a placebo or a competitor's ad instead — so you compare like with like. Randomized-controlled-trial work from large ad platforms found that without this correction, measured lift can be inflated several-fold, because the unobserved control isn't comparable to treatment.

The nuance: PSA controls solve comparability but introduce a different bias — the PSA itself occupies the slot and may depress baseline behavior. Ghost ads avoid that but require platform cooperation you may not have. And both still assume the experiment is large enough; intent-to-treat lift on rare conversions needs sample sizes most advertisers underestimate by an order of magnitude.

What to actually do: insist on a 'ghost' or intent-to-treat design from any platform selling you a lift study. Ask what the control group actually saw. If the answer is vague, the number is decorative.

<b>Bottom line for practitioners:</b> the validity of a lift test lives entirely in how the control was constructed. A clean control beats a clever model every time.