When to KILL a test early (my hard rules)
Took me 50 tests to stop being precious about losers. Sunk-cost kept me running dead experiments for weeks. Here's the kill list I follow now:
— Variant B has a real bug you just spotted. Kill instantly, fix, relaunch fresh.
— You changed the offer or price mid-test. Data's poisoned. Kill it.
— After full sample size, the gap is under 2% either way. It's a tie. Move on.
— Traffic dropped 40%+ from a source dying out. Sample's no longer representative.
— You realize the hypothesis was dumb. Yes, that's allowed. Kill it, learn, next.
What you DON'T kill for: 'it's been 3 days and B is losing.' Too early. Always too early.
Go look at your running tests. Anything matching rows 1-2? Kill it now.
Split Test Street
@SplitTestStreet
When to KILL a test early (my hard rules)
Этот пост опубликован в Telegram-канале Split Test Street. Подписаться можно по ссылке: @SplitTestStreet.