Pre-register your segments: a flat +0% test often hides +9% mobile / −7% desktop.

Pre-register your segments: a flat +0% test often hides +9% mobile / −7% desktop.
Aggregate results average away the truth. Split before you launch, not after.
— Declare 3-4 segments up front: device, new vs returning, traffic source, geo.
— Apply a correction for multiple comparisons (Bonferroni: divide α by number of segments).
— A post-hoc segment hunt across 12 cuts will always 'find' a winner — that's noise, not insight.
— Ship per-segment only where the effect is pre-registered and survives correction.
Slicing after the fact is how you fool yourself with a straight face. Read the number, not the story. [segments pre-reg=4]