<b>A variant can win every segment and still lose overall — Simpson's Paradox in funnels.</b>
If desktop and mobile each prefer B, but B happened to get more low-converting mobile traffic, the pooled number flips to A.
— Always check whether your randomizer balanced segment mix, not just totals.
— Report segment-weighted results, not raw pooled rates, when traffic mix differs by arm.
This is the same machinery as an SRM problem, one layer down.
Read the number, not the story. [segment-weighted · device split]
Conversion Lab Notes
@ConversionLabNotes
<b>A variant can win every segment and still lose overall — Simpson's Paradox in funnels.</b>
Этот пост опубликован в Telegram-канале Conversion Lab Notes. Подписаться можно по ссылке: @ConversionLabNotes.