Lock one primary metric before launch: testing 10 metrics at 95% gives a ~40% chance of a false flag.
More metrics is not more insight — it's more ways to be wrong by accident.
— Name exactly one primary metric, decided before traffic flows.
— Everything else is secondary (directional, never decision-grade).
— If you must judge several, correct α (Bonferroni: 0.05 ÷ k, or control false-discovery rate).
— A secondary metric crossing 95% is a hypothesis for the next test, not a result for this one.
A test with ten goals has no goal. Read the number, not the story. [primary=1, FWER controlled]
Conversion Lab Notes
@ConversionLabNotes
Lock one primary metric before launch: testing 10 metrics at 95% gives a ~40% chance of a false flag.
Этот пост опубликован в Telegram-канале Conversion Lab Notes. Подписаться можно по ссылке: @ConversionLabNotes.