Ramp new variants 1% → 5% → 50%, not straight to half your revenue. The goal of the early phase isn't power — it's catching a catastrophe cheaply. — Day 0-1 at 1%: watch error rate, latency, JS except…
Write the hypothesis card before building: tests with a pre-registered metric ship a decision 2x more often. The one-page template that prevents 'we're not sure what happened.' — Because we observed […
Triage session recordings in 30 minutes: watch the 12 sessions that rage-clicked, not 200 random ones. Qualitative review is a sampling problem. Sample the friction, not the average. — Filter recordin…
Prioritize funnel leaks by absolute lost users, not by step conversion rate. A 90% step and a 60% step can leak the same headcount — fix where the bodies are. — Map each step's entry count and exit co…
Isolate novelty effect: ~40% of early UI 'wins' decay to zero within two weeks among returning users. Returning visitors react to change itself, not to the change being better. The playbook: — Segment…
Run an A/A test before trusting your tooling: a healthy setup should show significance ~5% of the time, no more. If your platform 'finds' winners between two identical variants, every result is suspec…
