Clustering Keywords by SERP Overlap

Full transcript

1,143 words · script draft, under review.

Intro

You have three hundred keywords. You need to decide how many pages to build. One page per keyword is far too many. One page for everything is far too few.

So you start grouping by hand. "Espresso grinder" and "best espresso grinder" obviously belong together. "Burr grinder" and "conical burr grinder" too. You end up with thirty groups, and they feel right.

Then you build the pages. Some of them compete with each other in Google. Others never rank for half of the terms you put on them.

The problem is that you grouped keywords by how the words look. Google groups them by what searchers want. In this episode I'll show you how to cluster keywords using the search results themselves: if two keywords share enough of the same top ten URLs, they belong on one page.

Why word similarity isn't enough

There are two common ways to group keywords without looking at the results page. One is string similarity: shared words, shared word stems. The other is meaning: text embeddings, or asking a language model which terms belong together.

Both are fine as a first pass. And both fail in the same place.

Two phrases can share almost every word and still need different pages. "Espresso grinder" and "espresso grinder parts" share two words out of three. The first one wants a buying guide. The second wants a spare parts list.

And two phrases can share no words at all and want exactly the same page. "Coffee mill" and "coffee grinder" are the same thing to most searchers.

The search results page settles it. If Google shows mostly the same URLs for two queries, it treats them as the same need, and one page can rank for both. If the results are different, one page will struggle to cover both, however close the words look.

This also stops cannibalization before it starts. In episode twelve we found pages competing for the same query after the fact. Clustering by SERP overlap prevents it at the planning stage.

The method in four steps

The method has four steps.

Step one. Get the top ten organic results for every keyword. Only the organic URLs. Skip ads, maps and other features for this purpose. In this episode I use results pages I've already pulled and cached. How to pull them from the API, with location, device, depth and cost, is episode twenty-seven.

Step two. For every pair of keywords, count how many URLs they share in their top ten. That number is the overlap. Zero means nothing in common. Ten means identical results.

Step three. Pick a threshold. A good starting point is three shared URLs. Three or more, and the two keywords are linked. Fewer, and they're not.

Step four. Group the linked keywords into clusters. Each cluster becomes one page. The keyword with the highest volume becomes the main target. The others are supporting phrases that the same page should also cover.

One detail before you compare anything: normalize the URLs. Lowercase the host, strip tracking parameters, the trailing slash and anything after a hash sign. Otherwise the same page counts as two different URLs, and every overlap comes out too low.

Worked example

Here's an example, with made-up numbers for illustration. Eight keywords for a fictional coffee equipment store, example-shop.test.

The matrix shows the overlap for every pair. "Espresso grinder" and "best espresso grinder" share seven URLs. "Coffee mill" and "coffee grinder" share six. Those are easy. Same need, same page.

"Espresso grinder" and "espresso grinder parts" share one URL. Different page, just as we suspected.

"Burr grinder" and "espresso grinder" share three. Exactly on the threshold. That's the interesting case.

With a threshold of three, the eight keywords fall into three clusters: a grinder buying guide, a spare parts page, and a manual grinder page. With a threshold of four, "burr grinder" breaks off into its own cluster, and you'd plan four pages instead.

So which is right? Look at the shared URLs themselves. If the three URLs that the two keywords share are all broad buying guides, one page covering both is realistic. If they're three category pages from the same big retailer, the overlap tells you more about that retailer than about the searcher. In that case I lean towards a separate page.

The threshold is a starting point, not a verdict. Borderline pairs get a human look.

Four things that go wrong

Four things go wrong again and again.

First, chaining. If A links to B, and B links to C, simple grouping puts A and C in the same cluster even if they share nothing. With long chains you end up with one giant cluster that swallows half your list. The fix: require every keyword in a cluster to link to the cluster's main keyword, not just to some member. I build clusters around the highest-volume keyword first, then the next keyword that isn't assigned yet, and so on down the list.

Second, settings. A results page is a snapshot for one location, one language, one device, on one day. Only compare keywords whose results were pulled with the same settings, ideally in the same week.

Third, synonym groups. Remember episode twenty: when two variants show identical volumes, Google's data already treats them as one group. They will almost always land in the same cluster, and you must not add their volumes together when you calculate the cluster's demand. Count the group once.

Fourth, compare URLs, not domains. If a few big sites rank for everything in your niche, domain overlap will be high for every pair, and you'll get one cluster. Domain overlap tells you who dominates. URL overlap tells you which page satisfies the query.

The output: a cluster table

The output is a cluster table. One row per cluster: the main keyword, the supporting keywords, the cluster's total volume with synonym groups counted once, and the URLs that tie the cluster together.

That last column matters. When someone asks why two keywords share a page, you can show them the evidence. And when we get to SERP analysis in the next module, you already know which pages to study for each cluster.

In practice I let Claude Code write the overlap script. It reads the cached results from SQLite, normalizes the URLs, counts the shared URLs for every pair, and writes the clusters back to a table. It's a short script. The decisions that matter are the threshold and the chaining rule, and those are yours. Write them down next to the result, so you can rerun the same clustering in three months and compare.

Wrap-up

So: group keywords by what Google shows, not by how they're spelled. Start with three shared URLs, and check the borderline pairs by hand. The clustering script and the cluster table template are linked in the description. Next episode: how to break a big head term down into the phrases people actually type when they're ready to buy. See you there.