Sequential testing (always-valid p-values) lets you stop early without inflating false positives; fixed-sample does not.
Cost: sequential needs ~25% more total sample if the effect is small and you run to the end.
Fixed-sample is cheaper when you can pre-commit to a stop date and never peek. Almost nobody never peeks.
— Sequential ▮▮▮▮▮▮ peek-safety, ▮▮▮▮ efficiency
— Fixed ▮▮ peek-safety, ▮▮▮▮▮▮ efficiency
If your team checks the dashboard daily, sequential is not optional — it's the only honest math.
Read the number, not the story. [methodology · always-valid]
Conversion Lab Notes
@ConversionLabNotes
Sequential testing (always-valid p-values) lets you stop early without inflating false positives; fixed-sample
Этот пост опубликован в Telegram-канале Conversion Lab Notes. Подписаться можно по ссылке: @ConversionLabNotes.