CUPED variance reduction cuts required sample by ~20-50% when pre-period data correlates with the metric.
Reduction scales with correlation²; at r=0.6 you save ~36% of traffic.
CUPED (controlled pre-experiment data) regresses out predictable variance using each user's history. Useless for first-time visitors with no pre-period.
— Plain t-test ▮▮▮▮▮▮ sample needed
— CUPED ▮▮▮▮ sample needed
Use CUPED for logged-in/returning cohorts with rich history; skip it for cold acquisition traffic.
Read the number, not the story. [variance-reduction · r-dependent]
Conversion Lab Notes
@ConversionLabNotes
CUPED variance reduction cuts required sample by ~20-50% when pre-period data correlates with the metric.
Этот пост опубликован в Telegram-канале Conversion Lab Notes. Подписаться можно по ссылке: @ConversionLabNotes.