Geo holdout test: 'winning' channel had 0.3 incrementality
A team's MMP credited a retargeting channel with 28% of conversions. Last-touch made it the hero.
They ran a geo holdout (matched DMAs on vs off) for 3 weeks. Conversions in holdout regions barely moved — incrementality came out at ~0.3, meaning most credited conversions would have happened anyway.
They cut the channel's budget 60% and reinvested in prospecting; total installs held flat.
✓ Holdout exposed attribution claiming credit it didn't earn
✓ Freed 60% of a budget with zero volume loss
✗ Geo tests need scale and clean matched markets
✗ Three-week read is slow for fast-moving buyers
Verdict: last-touch credit ≠ incrementality — holdout your 'best' channel.
Best for: teams over-investing in retargeting.
In-App Bench
@InAppBench
Geo holdout test: 'winning' channel had 0.3 incrementality
Этот пост опубликован в Telegram-канале In-App Bench. Подписаться можно по ссылке: @InAppBench.