Howdy, Moz fans, and welcome to another edition of Whiteboard Friday! Today I’m going to talk about this extremely troublesome and worrisome problem that Google has expanded “keyword (not provided)” potentially to 100% of all organic referrals. This isn’t necessarily that they’ve flipped the entire switch, and everyone’s going to see it this week, but certainly over the next several months, it’s been suggested, we may receive no keyword data at all in referrals from Google. Very troubling and concerning, obviously, if you’re a web marketer.

I think it should be very troubling and concerning if you’re a web user as well, because marketers don’t use this data to do evil things or invade people’s privacy. Marketers use this data to make the web a better place. The agreement that marketers have always had—that website creators have always had—with search engines, since their inception was, “sure, we’ll let you crawl our sites, you provide us with the keyword data so that we can improve the Internet together. I think this is Google abusing their monopolistic position in the United States. Unfortunately, I don’t really see a way out of it. I don’t think marketers can make a strong enough case politically or to consumer groups to get this removed. Maybe the EU can eventually.

But in any case, let’s deal with the reality that we’re faced with today, which is that keyword not provided may be 100% of your referrals, and so keyword data is essentially gone. We don’t know when Google sends a visit—Bing, to their credit, and to Microsoft’s credit, enduringly has kept that data accessible—but we don’t know when Google sends a visit to our sites and pages, what that person searched for. Previously, we could do some sampling—now we can’t even do that.

There are some big tasks that we use that data for, and so to start with, I want to try and identify the uses for keyword referral data, at least the very important ones as I perceive them—there are certainly many more.

Number one: finding opportunities to improve a page’s performance or its ranking. If you see that a page of yours is receiving a lot of search traffic, or that a keyword is sending a lot of search traffic (or even a little bit of search traffic), but the page is not ranking very well, you know that by improving that page’s ranking you have an opportunity to earn a lot more search traffic. That’s a very valuable thing as a marketer. You can also see if a search query is sending traffic to a page, but that page has a high bounce rate for that traffic, low pages-per-visit, low conversion rate, you know, “hey, I’m not doing a good job serving the visitor; I need to improve how the page addresses that.” That’s one of the key things we use keyword referral data for.

Secondarily: connecting rank improvement efforts—things that we do in the SEO world to move up our rankings—to the traffic growth that we receive from them. This is very important for consultants and for agencies, and for in-house SEOs as well, to show our value to our managers, and our clients—it’s really, really tough to have this data taken away.

C: Understanding how your searchers perceive your brand and your content. When we look down the list of phrases that sent us traffic, we could see things like “oh, this is how people are thinking about my brand, or thinking about this product I launched, or thinking about this content that I’ve put out.” Really challenging to do that nowadays.

And D: uncovering keyword opportunities. We could certainly see, “this is sending a small amount of traffic, this is doing some long-tail stuff, hey—let’s turn this into a broader piece of content. Let’s try and optimize for some of those keyword phrases that we’re barely ranking on.” Or, we have a page that’s not really addressing that keyword phrase that we’re ranking on. We can address that. We can improve that.

So I’m going to try and tackle some relatively simplistic ways, and I’m not going to walk through all the details you would need to do this, but I think many folks in the SEO and marketing sphere will address these over the weeks and months to come.

Starting with A. How do I find opportunities to improve a page’s ranking or its performance with users when I can’t see keyword referral data? How do I know which page people are coming to? Thankfully, we can use the connection—the intersection of a few different sources of data. Pages that are receiving search visits is a big one, and this is going to be used throughout—instead of looking at keyword-level data, we’re going to be looking at page-level data. Which pages received referral visits from Google Search? Thankfully, that’s still data that we do get, and that’ll likely stay with us, because we can always see a referral source, and we know which pages are loaded. So, even if Google Analytics were to remove that, I think a third-party analytics provider would step in.

Pages receiving search visits plus rank-tracking data can get us a little close to this, because we can essentially say, “hey, we know this page is ranking well for these five or ten keywords that we have some reasonable expectation that they have keyword search volume. They’re receiving search visits, and yet they’re not performing well, or they’re not ranking particularly well, so improving them should be able to drive up our search traffic, improving their performance with users should be able to drive up our conversion rate optimization.

Optionally, we could also add in things like Google Webmaster Tools or AdWords data; AdWords data being used on they keyword side to fill in for, “hey, what’s the volume that a keyword is getting,” and Google Webmaster Tools data to be able to see a list of some keywords that maybe are sending us traffic. Dr. Pete wrote a good post recently about the relative accuracy of Google Webmaster Tools, and while unfortunately it’s not as good as any of the other methods, it’s still not awful, and so that data is potentially usable.

This will give us a list of pages that get search visits, or are targeting important search terms, that rank, and that have the potential to improve. So this gets us to the answer to this question. This used to be really simple to get at, now it’s more difficult, but still possible.

B. Connecting our SEO efforts to traffic growth from search. I know this is going to be tremendously hard, and this is probably one of the biggest tolls that this change is taking on SEO folks. Because as SEOs, as marketers, we’ve shown our value by saying, “look, we’re driving up search visits, some of it’s branded, some of it’s unbranded, some of it’s not provided—but you get a rough sense of this. And you really need that percentage: “What percent of the traffic is actually you going and getting us new visitors that never would have found us, versus branded stuff that’s just sort of rising on its own.” Maybe it’s rising because of efforts that marketers are making: investments in content, and in social media, and in email and all these other wonderful things, but it’s hard to know— it’s hard to directly map that.

So here’s one of the ways. Optionally, we can use AdWords to bid on branded terms and phrases. When we do that, you might want to have a relatively broad match on your branded terms and phrases so that you can see keyword volume that is branded from impression data. That gives you a sense of, “what’s the trajectory, here?” If we’re seeing it grow, we can identify “oh, that’s not us driving a bunch of new non-branded new keyword terms and phrases; that’s our brand search increasing.” So we can sort of discount that, or apply that in our reporting effectively. If we see, on the other hand, that it’s staying flat, but that search traffic overall is going up and to the right, then we know that’s unbranded.

Optionally, if we don’t want to be bidding and spending a lot of money with Google AdWords and trying to keep our impression counts high, we can use things like Google Insights or even downloading AdWords volume data estimates month-over-month to be able to track those sorts of things.

Certainly one of the things I would recommend doing even prior to this change is tracking rankings on buckets. Buckets of head terms, versus chunky middle, versus long-tail; so phrases that are getting lots of search volume, a good amount of search volume, and very little search volume. You want to have different buckets of those, so you can see, “oh hey, my rankings are generally improving in this bucket, or that bucket.” Same with branded vs. non-branded; you want to be able to identify and track those separately. Then, compare against visits that you’re seeing to pages that are ranking for those terms. We need to look at the pages that are receiving search traffic from those different buckets.

Again, much more challenging to do these days. But, any time we see the complexity of our practice is increasing, we also have an opportunity, because it means that those of us who are savvy, sophisticated, able to track this data, are far more useful and employable and important. Those organizations that use great marketers are going to receive outsized benefits from doing so.

C: How do I understand and analyze how searchers perceive my brand? What are they searching for that’s leading them to my site? How are they searching for terms related to my brand? Again, we can bid on AdWords terms, like I talked about. You can use keyword suggestion sources like Google Suggest, Ubersuggest, certainly AdWords’s own volume data, SEMRush, etc. to see the keyword expansions related to your brand or the content that’s very closely tied to your brand. And internal site search data. You’ve got a search box up in the top-right hand corner, people are typing in stuff, and you want to see what that “XYZ” is that they’re typing in. Those can help as well, and can provide you some opportunities that lead to D.

D: How do I uncover new keyword opportunities to target? Of course, there’s the classic methodology that we’ve all employed, which is keyword research, but usually we compare that to the terms that are already sending us traffic, and we go look and say, “oh, okay, we’re doing fine for these—we don’t need to worry.” Now, we need to take keyword research tools and add some form of rank-tracking data. That could be from Google Webmaster Tools despite its mediocrity in terms of accuracy. We can use manual rank data—we can search for it ourselves—or we can use automated data.

One of the criticisms for all rank-tracking data is always, “but there’s lots of personalization and geographic localization—these kinds of things that are biasing searches—how do I see all of that?” And the answer is, well, you can’t really. Personalization is going to fluctuate things. It may be sort of included in the Google Webmaster Tools data, but as Dr. Pete showed in his post, it looks a little funky right now.

For localization, you can add the geo in the string to be able to see where you rank in different geographies if you want to track those. That’s something you’ll be able to do in Moz Analytics and probably many of the other keyword tracking tools out there, too.

Optionally—and this is expensive, and I hate to say this is Google being evil, but this is probably what Google wants you to do when they give you “(not provided)”—which is run AdWords campaigns targeting those keywords, so that you can see new expansion opportunities. Areas where, “oh hey, we bid on this, it sent impressions, it sent some traffic, it looks like it’s worthwhile, we’re not ranking for it organically,” and again, you can see that through your rank-tracking data or through pages receiving visits from search, and then targeting those terms.

So, a lot of this data, and a lot of these opportunities are retrievable—they’re just a lot harder. I will say—this is somewhat self-promotional, but I think one of Moz’s missions and obligations as a company to the search marketing world is to try and help replace, repair, and make these processes easier. So, you can guess that over the next 6-12 months that’s going to be a big part of our roadmap: trying to help you folks—and all marketers—get to this data.

For now, these methodologies can and should be helpful to you, and I expect to see lots of great discussion about other ways to go about this in the comments.

Thanks, everyone—take care.