Platform-reported lift vs. independent experiments: who grades whose homework?

Platform-reported lift vs. independent experiments: who grades whose homework?

Meta says your campaign drove X conversions. Their Conversion Lift tool even shows a holdout. Should you trust the platform measuring its own effectiveness?

The structural conflict
Platform lift tools (Meta Conversion Lift, Google's experiments) often use sound methodology — ghost ads, randomized holdouts. The methodology isn't the problem; the incentive and the boundary are. The platform defines the conversion window, picks the attribution surface, and reports on itself. It cannot see conversions it didn't help cause but can claim ones that would have happened anyway if the window is generous.

Independent and platform-agnostic tests
A neutral geo experiment or MMM-calibrated lift study sits outside the walled garden. It can compare platforms on a common yardstick and detect cross-platform cannibalization that no single platform's tool will ever surface — because admitting Channel A stole Channel B's credit isn't in either platform's interest.

The nuance
This isn't an accusation of fraud. A well-run Conversion Lift test is genuinely informative about that platform in isolation. The failure is using ten such in-walled tests to allocate a budget across ten platforms — their windows overlap, so their summed "incremental" conversions exceed your actual total.

Bottom line for practitioners: Use platform lift tools for within-platform decisions (creative, audience, bid). Use independent, cross-platform experiments or MMM for between-platform budget allocation. The moment you're comparing vendors, never let a vendor be the referee — the conflict of interest is structural, not malicious.