Conversion Lab Notes
Conversion Lab Notes
@ConversionLabNotes

Myth: checking a test early doesn't hurt if you wait for significance. False — peeking quadruples your false-p

Myth: checking a test early doesn't hurt if you wait for significance. False — peeking quadruples your false-positive rate.
Fixed-horizon tests assume one look. Peek daily for 2 weeks and your true α drifts from 5% toward ~20–25%.
If you must monitor live, use sequential methods (mSPRT or always-valid p-values) built to survive continuous looking.
Fixed-horizon ▮▮ vs sequential-safe ▮▮▮▮▮▮
Read the number, not the story. [14 looks · α inflated 4–5×]
This post was published in the Telegram channel Conversion Lab Notes. Subscribe here: @ConversionLabNotes.
tech

Latest posts in “Tools and infrastructure”

All channels in this category →

start

Готовы запустить рекламу через сеть public.tg?

Новый оффер, продукт, GEO, кейс, событие или партнёрский запуск — соберём маршрут под задачу и отдадим медиаплан.

Telegram для медиаплана: @AFFtop_connect. Быстрый тест: $20 за канал, $1000 за пакет по сети.