Myth: high confidence means a big, real lift

People conflate two things and it costs them. Confidence (am I sure there's a difference?) is NOT effect size (how big is it?).

With enough traffic you can hit 99% confidence on a +0.3% lift. Statistically real. Practically pointless — won't survive your next deploy, and not worth the dev cost to ship and maintain.

Day 11 a test hit 99%. The lift? +0.4% on signups. I'd have spent a sprint shipping a rounding error.

Confidence tells you it's not zero. Effect size tells you if you should care. You need both.

Go set a MINIMUM lift you'll bother shipping (mine's 3%) before you start. Below it, walk away even at 99%. Report back.