<b>30% of my test traffic wasn't human</b>
Day 7: a test that made no sense. Numbers all over the place, conversion rate weirdly low on both sides.
Dug into the logs. Bots. Scrapers, headless crawlers, a competitor's monitoring tool, all getting bucketed into my test and never converting. They diluted everything and added pure noise.
On a smaller offer page, bot traffic can be 20-40% of hits. It doesn't bias toward A or B, but it inflates your sample with zeros and slows your time-to-significance to a crawl.
Fix: filter known bots and direct-no-referrer junk BEFORE bucketing.
— Check what % of your test traffic converts at literally 0
— Cross-ref with your bot/crawler logs
Go add a bot filter before your test bucketing. Report back.
Split Test Street
@SplitTestStreet
<b>30% of my test traffic wasn't human</b>
Этот пост опубликован в Telegram-канале Split Test Street. Подписаться можно по ссылке: @SplitTestStreet.