Switching your primary metric after seeing results turns a null test into a 20-30% "win" rate by chance alone.

Switching your primary metric after seeing results turns a null test into a 20-30% "win" rate by chance alone.
The failure: CVR (conversion rate) was flat, but add-to-cart moved, so you report that instead.
— Each metric you scan is another roll of the dice. Five metrics, one will likely cross p<0.05 on noise.
— Fix: declare one primary metric and the direction before launch. Guardrails are secondary, never promoted to the headline.
Read the number, not the story. [5 metrics · ~23% spurious]