<b>Counting conversions per session instead of per user can swing a result by 10%+.</b>

<b>Counting conversions per session instead of per user can swing a result by 10%+.</b>
Users with multiple sessions get counted multiple times, and heavy users skew toward whichever arm they landed in more.
— Pick the randomization unit (user) and the analysis unit (user) to match. De-duplicate.
— Session-level conversion rates flatter high-frequency cohorts and break independence assumptions.
The denominator decides the answer before the variant does.
Read the number, not the story. [unit: user · de-duped]