Measured across parallel tracking on 11 tests; gap driven by attribution windows, bot filtering, and dedup logic.
Neither is "right." The A/B tool counts within the experiment's randomization unit; analytics counts on its own session model. Mixing them invalidates the comparison.
— Pick one source of truth ▮▮▮▮▮▮
— Average two tools ▮ (do not do this)
Decide the metric system before the test starts, then never cross-check mid-flight.
Read the number, not the story. [n=11 tests · ~8% gap]
Conversion Lab Notes
@ConversionLabNotes
Your analytics platform and your A/B tool will disagree on conversions by a median ~8%.
This post was published in the Telegram channel Conversion Lab Notes. Subscribe here: @ConversionLabNotes.