Myth: a 95% confidence result is 95% likely to be true. False — that's not what the number measures.
p<0.05 means: if there were no real effect, you'd see data this extreme under 5% of repeats. It says nothing about the odds your hypothesis is correct.
With a low base rate of real winners (say 1 in 10 tests), most 'significant' wins are still false. Posterior, not p-value, is the question.
Read the number, not the story. [α=0.05 ≠ P(true)]
Conversion Lab Notes
@ConversionLabNotes
Myth: a 95% confidence result is 95% likely to be true. False — that's not what the number measures.
Этот пост опубликован в Telegram-канале Conversion Lab Notes. Подписаться можно по ссылке: @ConversionLabNotes.