A checklist to keep geo experiments from contaminating each other
The question: you're running multiple geo tests at once to move faster. But overlapping experiments can poison each other's controls, turning rigor into noise. How do you keep them clean?
The anti-contamination checklist:
— Map spillover geography first. Adjacent markets share media (regional TV, metro radio) and shoppers cross borders. A control market that sees your test market's billboards isn't a clean control.
— Enforce buffer regions. Leave unused geos between test and control to absorb spillover, the way field trials use buffer plots.
— Don't let two tests share a control pool. If channel A's test geos are channel B's controls, each test's intervention contaminates the other's baseline.
— Stagger start dates so carryover (adstock) from a prior test has decayed before the next begins — often 2-4 weeks of washout.
— Pre-register which geos belong to which test, locked before launch.
The nuance: contamination biases toward understating lift — when your control market is partly treated, the measured gap shrinks and a real effect looks null. Teams then wrongly conclude a working channel doesn't work. The error is silent because the math still runs cleanly.
Bottom line for practitioners: map spillover, build buffers, isolate control pools, and stagger with washout periods before running concurrent geo tests. Parallel testing is powerful, but an experiment whose control is secretly contaminated gives you a precise, confident, wrong answer.
Credit Where Due
@CreditWhereDue
A checklist to keep geo experiments from contaminating each other
Этот пост опубликован в Telegram-канале Credit Where Due. Подписаться можно по ссылке: @CreditWhereDue.