Running 4 overlapping tests on the same checkout assumes additivity — and interactions can erase 30% of stacked gains.
The error: summing four individual +3% wins to forecast +12%. Effects aren't independent when they touch the same flow.
— Two friction-reducers can't each "remove the same friction."
— Fix: use mutually exclusive traffic buckets for conflicting tests, or a factorial design that estimates the interaction term directly.
Read the number, not the story. [Σ wins ≠ combined · interaction]
Conversion Lab Notes
@ConversionLabNotes
Running 4 overlapping tests on the same checkout assumes additivity — and interactions can erase 30% of stacke
Этот пост опубликован в Telegram-канале Conversion Lab Notes. Подписаться можно по ссылке: @ConversionLabNotes.