"I changed the titles and traffic went up, so the change worked."

"I changed the titles and traffic went up, so the change worked."

No. That's a before/after, and before/after is the weakest tool you own.

Here's the comparison that matters. A before/after measures your change plus the algorithm update plus seasonality plus whatever your competitor did that week. You can't separate them. A proper split test — split your URLs into matched groups, change one group, leave the other as control — measures the change alone.

Use before/after for sitewide moves you can't partition (a CMS migration, a global template swap). You have no choice; just don't pretend it's causal.

Use split testing for anything templated and repeatable: title patterns, schema additions, intro length. Tools like the SearchPilot approach exist precisely because "traffic went up" is not evidence.

The tell: ask "what did the pages I didn't touch do?" If you don't have that number, you ran a story, not a test.

(November traffic going up means nothing. November traffic going up versus a held-out control means something.)