Sophie Brannon lays out four ways to prove an SEO change worked

A manager asks why a page rose or fell. The usual answer is a chart of traffic before and after a change was made. That chart cannot tell the change apart from the Google update that landed the same week.

Sophie Brannon writes in Search Engine Journal that SEO teams get around this with proper split testing. Brannon is co-founder and director at StudioHawk US. The piece was published on 9 November. The method is ordinary A/B testing, applied to pages a search engine crawls rather than to buttons a user clicks.

Four designs Brannon lists

Brannon lists four designs, ordered from strongest to weakest.

Page-split testing takes template pages. Some are held back as a control. The rest get the change. The test and the control run over the same period, so they share the same Google weather. Geo-split testing runs the change in some markets and not in others. Markets without the change act as the control. Modelled baseline uses external data from a place where the change could not have happened. Pre and post comparison is the weakest. It cannot separate the change from the other things happening on the site.

She names a scale. About 1,000 organic visits a day on the test pages is her floor for a trustworthy result.

Tools and numbers she cites

Brannon names two tools. Google's CausalImpact is an open-source R package that builds a forecast of what traffic would have been without the change. Teams can read the gap between forecast and actual as the effect. SearchPilot is a paid platform that runs SEO split tests with a neural network as the forecast model.

She cites a SearchPilot figure. The firm re-ran historical tests and found that 29 percent of them held up at the 95 percent confidence line. That is SearchPilot's own measurement on its own tests.

She also gives a worked example of a test result: a 14.2 percent click increase with a credible interval of 8 to 19 percent at above 99 percent probability.

What to write down before the test starts

Brannon's advice is to write a one-page test brief and share it before the change goes live. The brief holds the hypothesis and the primary metric. It names the guardrails and the minimum useful lift. It fixes the date the result is read and the rule for a pass or a fail. Decisions decided in advance stop the test from becoming a story told after the fact.

For a Reveneau reader who runs SEO for a product, the useful idea is the control group. A page sits beside the changed page, under the same Google weather. The manager who asks whether a drop was the team or the algorithm now has a real answer. A chart on its own cannot answer that.

Source

SEO Split Testing In 2027: How To Prove A Change Actually Worked (Search Engine Journal)