Playbook: Run a clean RPM A/B test

Playbook: Run a clean RPM A/B test
What it does: splits traffic between two layouts to find the real revenue winner, not a vibe.

Setup (effort 3/5):
— Split at the user level (cookie/localStorage bucket), not per-pageview — same reader, same layout.
— Hold everything else fixed: same content, same period, same traffic mix.
— Run minimum 7 days to cover weekday/weekend CPM swings. Two days proves nothing.
— Compare session RPM and bounce together — a layout that lifts RPM but spikes bounce loses long-term.
— Need ~10k sessions/arm for a stable read on a 5% difference. Smaller = noise.

RPM lift seen: the test itself doesn't lift — it prevents you shipping a 3% loser you'd have called a winner.

Pros: kills guesswork. Cons: slow, needs traffic volume, easy to misread underpowered tests.

Lift: ★★★★☆ — Risk: ★☆☆☆☆
Best for: 50k+ sessions/month. Skip below that — you can't reach significance.