<b>Case #017: The $50 that lied about everything</b>

<b>Case #017: The $50 that lied about everything</b>

A cautionary arc about killing offers too fast on too little data. We tested eight offers in a vertical, $50 each, planning to scale the winner and bury the rest. Clean methodology. Completely broken in practice.

— Offer A: $50 spent, 2 conversions, looked like a winner at 60% ROI
— Offer F: $50 spent, 0 conversions, declared dead

We scaled A, buried F, moved on. Offer A collapsed to -20% within three days at higher budget. Offer F, which a colleague resurrected out of stubbornness, settled at a steady 40% ROI once it had real volume.

The problem was statistics. At $50 and a $12 payout, a "winning" offer was decided by literally two conversions. Two. Random chance — one lucky high-intent visitor — was enough to crown a loser and condemn a winner. We'd built a tournament where the result was noise.

— Conversions needed for a stable read at our payout: roughly 15-20
— Conversions we judged on: 0 to 3

We re-ran the test with a real floor: no offer gets killed before 50 clicks AND $150 spent, whichever comes later, unless it's catastrophically dead (zero conversions at high click volume). The verdicts inverted on three of the eight.

— Test round one cost: $400 and two wrong decisions
— Test round two: $1,200, but every verdict held when scaled

The lesson: a $50 test on a $12 offer is decided by luck, not signal — give each test enough budget to produce 15+ conversions, or you'll scale noise and bury winners.