Bayesian tests reach a decision ~22% faster than fixed-horizon frequentist on the same traffic.
Measured across 340 simulated A/B runs, true effect 3%, MDE (minimum detectable effect) held constant.
The catch: Bayesian "95% probability B beats A" is not a guaranteed false-positive rate. Peek freely with Bayesian; with frequentist, peeking inflates error.
— Frequentist ▮▮▮▮▮▮ days vs Bayesian ▮▮▮▮▮ days
— Use frequentist when stakeholders demand a fixed p-value gate; Bayesian when you need to ship and iterate.
Read the number, not the story. [n=340 sims · 95%]
Conversion Lab Notes
@ConversionLabNotes
Bayesian tests reach a decision ~22% faster than fixed-horizon frequentist on the same traffic.
Этот пост опубликован в Telegram-канале Conversion Lab Notes. Подписаться можно по ссылке: @ConversionLabNotes.