Posted by Dr. Pete
As you start tracking your rankings and taking SEO more seriously, you're bound to ask the question (and we hear it a lot) – "What are ALL of the keywords that my site ranks for?" Sounds simple enough, but it turns out this question isn't just complicated – it's probably unanswerable.
I'm going to walk you through why it's such a tough question, discussing two myths that lead us to ask it in the first place. Then, I'm going to try to at least give you a partial answer – maybe not all, but enough to keep you busy for a long time.
Myth #1 – The Ranking Table
If you have any experience with programming, databases, or even just Excel, it’s pretty easy to envision Google as some kind of giant ranking table. It might look something like this:
| Keyword | Site | Rank |
|---|---|---|
| unicorns | www.unicorns.com | #1 |
| unicorns | www.bobsunicornshack.com | #2 |
| unicorns | www.unicornwrestling.org | #3 |
| pretty unicorns | www.ohsoprettyunicorns.com | #1 |
| pretty unicorns | www.sexymalibuunicorns.com | #2 |
| pretty pretty unicorns | www.examplesgonetoofar.com | #1 |
While this approach might work for a very basic, closed system (like an internal knowledge base), it’s not remotely practical on the scale of something like Google. The sheer scope of data, the blinding speed it gets updated, and the way that data has to be distributed across server farms (made up of thousands of servers), means that modern search is essentially a real-time calculation. There is no master table.
Myth #2 – Google Won't Tell Us
Ok, so it’s not a table, but Google still knows what we rank for or they could figure it out, right? While Google definitely has plenty of data they won’t let us see, some things are mysteries even to them. Back in 2007, Google’s VP of Engineer, Udi Manber, shocked the search community by suggesting that as many as 20-25% of all Google queries were queries they had never seen before. Let's say that again – as many as 1/4 of all Google searches are new. Google later clarified that this is within a time window (not all of search history), but the number is still staggeringly high.
Much of this has to do with the fact that queries are naturally getting longer and more specific, with over half of search queries in 2010 being 4 words or longer. As people get more comfortable with asking detailed, natural-language questions, this trend is only going to continue. One way or another, your site is ranking for new keywords every day, and some of them are a surprise even to Google.
Tactic #1 – Mine Your Analytics
So, is figuring out what you rank for as elusive as the unicorns in my table? Fortunately, no. While you'll never know ALL of the keywords you rank for, you can definitely find a solid pile of data. Your best, first destination is your own analytics – here's an example from Google Analytics (go to "Traffic Sources" > "Keywords > "Non-paid"):
Of course, these are only keywords that drove clicks, but for my own site this represents 1,435 keyword phrases in just 1 month. My blog is hardly exceptional – it gets just over 200 visitors per day. So before you dismiss your analytics because they don't show you EVERYTHING, ask yourself if you've even come close to using the data they do provide.
Tactic #2 – Review GWT Keywords
The second place to look for keywords you're ranking for is Google Webmaster Tools, which is one of the only places to see data for keywords that drive search impressions but NOT clicks. Within GWT, go to "Your site on the web" > "Search queries", and you'll see something like this:
The "Clicks" column actually only goes down to "<10", so it's difficult to tell exactly which keywords drove no clicks, but comparing this data to your analytics data can help fill in some of the holes, if you really want to see the big picture.
Tactic #3 – Analyze Inbound Anchor Text
So, what if you want to find keywords that people aren't currently searching for but for which you could potentially rank? One place you might look is the anchor text that external sites use to link to your site, especially the longer tail phrases. For example, in our own Open Site Explorer, click on the "Anchor Text Distribution" tab and you'll get a full list of the phrases or terms external sites use to link to you (export to excel for up to 10,000 results):
For example, I would rank #1 for "muppet intern yoozer", if anyone actually ever typed that phrase (before I did today). I'm not sure how that helps me, but at least conceptually, seeing what phrases people are using to link to you can give you a sense of what you have the capacity to rank for, even if those phrases don't currently drive searches.
Stop Obsessing & Get to Work
So, maybe you can’t find ALL the keywords you’d ever rank for, but so what? Using these techniques and extrapolating a bit (put in some quality time with Excel), you can easily generate a list of hundreds or thousands of keywords that you either currently or could potentially rank for. That ought to keep you busy for a while.