Lessons Learned: 10 Mistakes in SEO Data Analysis

Full transcript

985 words · script draft, under review.

Intro

This is the fiftieth and last episode of the series. And I want to end with the most useful thing I can give you: the mistakes.

Not theoretical ones. Almost all of these I've made myself, on my own sites, with real money and real time spent on the wrong conclusion. Some of them I only found because a number looked a little too good.

In this episode: ten mistakes in SEO data analysis, what each one looks like, how to catch it, and which episode covers the fix.

Mistakes one to three: measuring the wrong thing

Mistake one: not filtering Analytics by hostname. An Analytics property can outlive a domain change, and it keeps collecting whatever is sent to it. I had a dashboard showing over fifty thousand sessions for a site. Only about fifty-five came from the current domain. The rest was bot spam on a retired domain. The fix is one filter, applied to every query, from episode fifteen.

Mistake two: trusting the "indexed" number in the sitemap report. In my experience it simply doesn't match reality. Real index status comes from URL Inspection, on a stratified sample, or from exact-phrase searches. And even URL Inspection is reliable in aggregate, not per URL, so always run a known-good control site in the same check. Episode seventeen.

Mistake three: counting crawling in your server log when you're behind a CDN. The origin server only sees requests the cache didn't answer. When I measured it, the origin saw about five percent of Googlebot's requests. Your crawl numbers live at the CDN, not on your server.

Mistakes four and five: trusting the label

Mistake four: identifying bots by user agent. The user agent is just a text string, and anyone can send "Googlebot". In my measurement, about fifty-six percent of requests claiming to be Googlebot were fake. Verify by IP address against Google's published ranges. Never by the name a visitor gives itself.

Mistake five: analysing from one data source. Search Console tells you what happened, but not what Google showed instead of your page. A keyword tool tells you volume, but not whether it's winnable. That's the whole reason the framework from episode one has three parts. Each source answers a question the others can't. A typical example: clicks fall on a page, and Search Console alone says the ranking is unchanged. Only the results page shows the reason, a new feature above the ten blue links taking the clicks. One source would have sent you rewriting a page that was never the problem.

Mistakes six and seven: breaking the keyword data

Mistake six: summing search volumes that are really one group. When several variants of a keyword show exactly the same volume, that's usually Google grouping them as one. Add them up and you triple your demand on paper. Equal volumes mean one synonym group. Count it once. Episode twenty.

Mistake seven: picking the wrong keyword form. Singular and plural can have very different volumes, and the form that wins nationally isn't always the form that wins at a regional level. Run the whole dataset on the wrong form and every number in it is off. The cure is cheap: a small validation task on both forms, at both levels, before any big run. A related trap: a "zero" volume usually means below the measurement floor of about ten searches a month, not zero demand.

Mistakes eight to ten: no baseline, fake precision, no follow-up

Mistake eight: no baseline. You ship a change, traffic moves, and you have nothing stored from before to compare with. Was it the change, the season, or an update? Without a baseline and a change log, you can't answer. Store the numbers before you touch anything. Episodes thirteen and forty-six.

Mistake nine: fake precision in scoring. A score of point seven three four looks scientific. But it's built from estimated volumes, estimated click-through rates and judgement calls about winnability. Use bands, high, medium, low, and let dependencies and effort decide the order. Episodes forty-three and forty-four.

Mistake ten: never re-measuring. The most common one, and the most expensive. If you don't check whether a fix worked, against control pages, in a proper window, you can't learn. You just keep doing what feels right. And every unmeasured fix makes the next analysis harder, because you no longer know which of your changes are still doing anything. Put the re-measure date in the plan the day you decide. Episode forty-six.

The pattern behind all ten

Look at the list again and you'll see a pattern. Not one of these mistakes is about a clever technique. They're all about trust.

The first four are measurement mistakes: the number isn't counting what you think it counts. The next three are data mistakes: the dataset is wrong before the analysis even starts. The last three are about the loop: you can't tell whether anything you did made a difference.

So the habit that prevents most of them is the same. Before you believe a number, ask where it came from, what it really counts, and what you would compare it with.

Closing the series

That brings us back to where we started. In episode one, I gave you a framework: traffic, then the results page, then the gap. Fix, and re-measure. Fifty episodes later, that loop is still the whole method. Everything else in the series is a way to do one of those steps more carefully.

If you're new here, start with episode one and run the loop on a site you own. If you've followed along, run it again in three months. Your second pass will find things your first one missed.

Thank you for watching, all fifty of them or just this one. Building this series has made my own analysis better, and many of your comments and questions found their way into the episodes.

Wrap-up

Ten mistakes, one habit: check where every number comes from. The checklist of all ten, with the episode for each fix, is linked in the description, along with the full playlist. The best next step is episode one, with your own data. Thanks for being here.