Set MDE before you launch, not after: 80% of inconclusive tests were underpowered from day one.
The one-task playbook: size the test so you don't waste 3 weeks.
— Pull baseline conversion rate from 28 days of traffic (your p, the proportion that converts).
— Pick MDE (minimum detectable effect — smallest lift worth shipping). Below 2% relative on a checkout? Usually not worth it.
— Plug p, MDE, 80% power, 95% confidence into any sample calculator → required n per arm.
— Divide n by daily traffic-per-arm. If runtime > 4 weeks, raise MDE or kill the idea now.
Low baseline + small MDE = traffic you don't have. Read the number, not the story. [power=80% · 95%]
Conversion Lab Notes
@ConversionLabNotes
Set MDE before you launch, not after: 80% of inconclusive tests were underpowered from day one.
Этот пост опубликован в Telegram-канале Conversion Lab Notes. Подписаться можно по ссылке: @ConversionLabNotes.