Ignoring pre-experiment data leaves ~20-50% of achievable power on the table.
The mistake: comparing raw means when each user's prior behavior is a strong covariate you could regress out.
— CUPED (controlled-experiment using pre-experiment data) uses pre-period metrics to shrink variance, often cutting required sample by a third.
— Fix: adjust the metric by its pre-period value before the t-test. Same traffic, tighter confidence interval.
Read the number, not the story. [variance −20-50% · CUPED]
Conversion Lab Notes
@ConversionLabNotes
Ignoring pre-experiment data leaves ~20-50% of achievable power on the table.
Этот пост опубликован в Telegram-канале Conversion Lab Notes. Подписаться можно по ссылке: @ConversionLabNotes.