The cumulative conversion-rate chart always looks "converged" — and that look fools teams into early stops.
The trap: cumulative curves smooth out by construction; early noise gets diluted by later data, so the line flattens before significance is real.
— It creates false confidence the gap is "stable."
— Fix: ignore the cumulative line for decisions. Use the confidence interval on the final pooled estimate and the pre-set sample size.
Read the number, not the story. [cumulative ≠ converged]
Conversion Lab Notes
@ConversionLabNotes
The cumulative conversion-rate chart always looks "converged" — and that look fools teams into early stops.
Этот пост опубликован в Telegram-канале Conversion Lab Notes. Подписаться можно по ссылке: @ConversionLabNotes.