Bayesian vs frequentist: stop fighting, here's when
Day 4 of a test, traffic's low, and the team wants an answer. Classic.
Frequentist (p-value, fixed sample): great when you have big traffic and patience. Punishes peeking. You WILL peek. You'll lie to yourself.
Bayesian (probability to beat baseline): lets you peek without inflating false positives as badly, gives you '87% chance B wins.' Decisions feel human.
My rule from the trenches:
— High traffic, one critical test → frequentist, set sample size, don't touch it.
— Low traffic, many scrappy tests → Bayesian, accept the fuzziness, move fast.
Lost a test last month because I called a Bayesian 80% as a win. It wasn't. 80% means 1-in-5 you're wrong.
Go find out your tool's default. Report back.
Split Test Street
@SplitTestStreet
Bayesian vs frequentist: stop fighting, here's when
Этот пост опубликован в Telegram-канале Split Test Street. Подписаться можно по ссылке: @SplitTestStreet.