A "winning" test at 40% power has a ~40% chance the true effect points the other way.

A "winning" test at 40% power has a ~40% chance the true effect points the other way.
Underpowered tests don't just miss winners — they produce winners that reverse on rollout (the Type-S, or wrong-sign, error).
— The trap: tiny sample, big observed lift, you ship it, traffic-wide it's flat or negative.
— Fix: power to 80%+ before calling anything. Compute MDE (smallest effect you can detect) first; if it's 9% and you expected 3%, you're testing nothing.
Read the number, not the story. [power 40% · sign-error 40%]