Beyond ~4 weeks, cookie deletion inflates new-visitor counts, and promos/seasonality shift the baseline mid-test, biasing the comparison.
Size for power, run full business cycles (capture weekday/weekend), then stop. Indefinite runs aren't rigor — they're undefined stopping rules.
Read the number, not the story. [2–4 wk · full cycle, then stop]
Conversion Lab Notes
@ConversionLabNotes
Myth: longer tests are always safer. False — over-running invites drift and seasonality contamination.
This post was published in the Telegram channel Conversion Lab Notes (@ConversionLabNotes). All its posts are on the channel page.
Open on Telegram