<b>One whale order can manufacture a fake 8% revenue lift in a mid-size test.</b>

<b>One whale order can manufacture a fake 8% revenue lift in a mid-size test.</b>
Revenue is heavy-tailed; the mean is hostage to extremes. A single $9,000 order in the variant arm moves the average that the t-test reads as "signal."

— Winsorize or cap order value, or test median / trimmed mean.
— Always eyeball the per-arm max order before believing a revenue win.

Raw mean ▮▮▮▮▮ vs Trimmed ▮▮ — same data.

Read the number, not the story. [99th-pctile cap]