Posted by SteKenwrightThis post was originally in YouMoz, and was promoted to the main blog because it provides great value and interest to our community. The author’s views are entirely his or her own and may not reflect the views of SEOmoz, Inc.
Thi… Continue reading →
Posted by Ryan_Watson
Did you know that there are over 90 billion URLs are packed into our Mozscape API? That’s a lot of links. So many links, in fact, that it can be daunting to dream up all of the many ways that you could put those links to good use. When we originally built Linkscape (the predecessor to Mozscape), we mainly had one thing in mind… SEO and backlinks.
But there’s a whole lot more than that.
Links are only the beginning, it’s what those links can tell us that’s so darn interesting. Which is why I wanted to call out all of the amazing ways that developers (and marketers) are using Mozscape data to better their work, as well as encourage new uses of Mozscape data that have yet to be explored. (Feel free to jump in and create your own API key any time.)
Mozscape’s wealth of links can be used in a variety of ways: from SEO audits, to domain valuations, to excel integration. Here at Moz, we have only begun to scratch the surface of how we can utilize the API. We currently use it to run some of our own tools such as Open Site Explorer and the Mozbar.
But I don’t want to focus on the way we use it. Let’s take a look at the way other developers have demonstrated some exciting uses for Mozscape. Hopefully these will get your mind going, thinking up other ways to use the data as well.
We’ll start with the most obvious of use cases, SEO audits. There quite a few examples of SEO audit tools that use Mozscape data, but a few of our favorites (that are in front of a paywall) are the HubSpot Website Grader and The Found SEO Audit Tool, both of which bring the heat.
Mozscape data is what powers things like the total pages indexed by search, MozRank, a list of the most authoritative pages, along with their corresponding anchor texts. The beauty of this use case is that it can provide a great lead-gen funnel for all of the SEO agencies out there, proving value up front with an email address required prior to running the report. As a digital marketing agency, using Mozscape data to develop a site audit is a great way to get users into your sales funnel. You know, that inbound marketing stuff — cold calls are old news.
How valuable is a website, purely from an online authority perspective? Traditionally, that was a very tough question. You could look at things like site traffic (which typically isn’t very accurate) or rankings for certain terms, but that’s a far-sighted approach to the question. Think about using the metrics behind Mozscape, like MozRank, Domain Authority, and MozTrust instead. Flippa, for example, uses Mozscape data as a datapoint for due diligence.
You could imagine this kind of domain valuation anywhere else domains are bought or sold, most of which have yet to use Mozscape data. The value, of course, is providing as much confidence to the buyers of web properties based on the “web footprint” of the site.
The spreadsheet kung-fu of this industry is unmatched anywhere else. With the integration of Mozscape data to Excel, some have been able to make Excel sing. The beauty of using Excel for analyzing Mozscape data is that you can slice and dice as you please, without setting up complex API calls. Perhaps our favorite example of Excel comes from the illustrious Richard Baxter, with the Links API Extension from SEO Gadget.
However, if Google Docs are more up your alley, the amazing Aleyda Solis created just the thing for you (so did Chris Lee). Tools like these allow the average marketer to dig into the firehose of data available through the API in a simple and recognizable interface.
Yes, that’s right. iAcquire uses the data when creating client reports as it not only helps them to inform the client about how their pages are doing but to also show the importance of certain pages on their site. The data is both a research tool and an education tool.
“Below is a screenshot from a ranking research report showing data we gathered for the keyword ‘inbound marketing tips.’ Moz stats are represented throughout the stats columns. As we work with these reports we are able to see if any of our content distribution efforts resulted in links on page or domain as can be seen in the far left columns.”
That’s how Mozscape is being used today, but it’s only the tip of the iceberg. A few folks have realized the potential outside of the traditional use cases that I’ve mentioned above. The power of the data comes when we take Mozscape data outside of its traditional context of pure link evaluation. Let me show you what I mean.
Its relatively easy to imagine Mozscape’s data being used for link building. With Mozscape’s massive amount of link data, SEOs are able to prioritize their link building efforts, and focus on value added efforts.
You could imagine that some of the examples noted above have been used for link building, but what about a deeper integration into a contact manager? Something that would allow the user to prioritize outreach by the value of a domain.
Just as one can do with the Klout score (or Social Authority) on Twitter, the same can be done for customer relationship efforts in filtering Domain Authority to determine importance.
We’ve seen hints of blogs using Mozscape data determine a top startup list, like the GeekWire 200, but the same could be applied for any rankings list of web properties.
Traditionally, lists have used Alexa or Compete traffic data to determine web prominence, but they’re so inaccurate. Other lists have used social specific metrics like social followings, but those too fall short. Geekwire’s list of the top 200 startups in Seattle uses a blend of both social and web data (External links, MozTrust) to determine just how influential a site is, providing the full picture.
I’m sure we’ve missed a ton of ideas, so we’re calling on you to help us find those new opportunities for Mozscape. Things like a tightening relationship between links and social networks, and categorizing link sources. How would you use this data, and how would you build it? Better yet, why not create your key and get going?
We want to make it easy for you.
We’ve been working quite hard to make our indexes faster and have recently updated our Mozscape API documentation. We want to make it as simple for you to use the data to get your idea up an running as possible.
Plus, if you create something, it’s likely we’ll get you added to our app gallery. We have everything from large corporations to individuals who have used the API and we show off their work in the gallery.
We’d love to hear from you. Obviously we always encourage folks to jump in and check out the free API (as well as the paid), and use the data for something useful for you. We’re also quite open to hearing about ways we can improve our own tools with the data or help educate people better. I look forward to reading through your feedback and seeing if there are ways we can help get people started using Mozscape.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!
Posted by Ryan_Watson
Did you know that there are over 90 billion URLs are packed into our Mozscape API? That’s a lot of links. So many links, in fact, that it can be daunting to dream up all of the many ways that you could put those links to good use. When we originally built Linkscape (the predecessor to Mozscape), we mainly had one thing in mind… SEO and backlinks.
But there’s a whole lot more than that.
Links are only the beginning, it’s what those links can tell us that’s so darn interesting. Which is why I wanted to call out all of the amazing ways that developers (and marketers) are using Mozscape data to better their work, as well as encourage new uses of Mozscape data that have yet to be explored. (Feel free to jump in and create your own API key any time.)
Mozscape’s wealth of links can be used in a variety of ways: from SEO audits, to domain valuations, to excel integration. Here at Moz, we have only begun to scratch the surface of how we can utilize the API. We currently use it to run some of our own tools such as Open Site Explorer and the Mozbar.
But I don’t want to focus on the way we use it. Let’s take a look at the way other developers have demonstrated some exciting uses for Mozscape. Hopefully these will get your mind going, thinking up other ways to use the data as well.
We’ll start with the most obvious of use cases, SEO audits. There quite a few examples of SEO audit tools that use Mozscape data, but a few of our favorites (that are in front of a paywall) are the HubSpot Website Grader and The Found SEO Audit Tool, both of which bring the heat.
Mozscape data is what powers things like the total pages indexed by search, MozRank, a list of the most authoritative pages, along with their corresponding anchor texts. The beauty of this use case is that it can provide a great lead-gen funnel for all of the SEO agencies out there, proving value up front with an email address required prior to running the report. As a digital marketing agency, using Mozscape data to develop a site audit is a great way to get users into your sales funnel. You know, that inbound marketing stuff — cold calls are old news.
How valuable is a website, purely from an online authority perspective? Traditionally, that was a very tough question. You could look at things like site traffic (which typically isn’t very accurate) or rankings for certain terms, but that’s a far-sighted approach to the question. Think about using the metrics behind Mozscape, like MozRank, Domain Authority, and MozTrust instead. Flippa, for example, uses Mozscape data as a datapoint for due diligence.
You could imagine this kind of domain valuation anywhere else domains are bought or sold, most of which have yet to use Mozscape data. The value, of course, is providing as much confidence to the buyers of web properties based on the “web footprint” of the site.
The spreadsheet kung-fu of this industry is unmatched anywhere else. With the integration of Mozscape data to Excel, some have been able to make Excel sing. The beauty of using Excel for analyzing Mozscape data is that you can slice and dice as you please, without setting up complex API calls. Perhaps our favorite example of Excel comes from the illustrious Richard Baxter, with the Links API Extension from SEO Gadget.
However, if Google Docs are more up your alley, the amazing Aleyda Solis created just the thing for you (so did Chris Lee). Tools like these allow the average marketer to dig into the firehose of data available through the API in a simple and recognizable interface.
Yes, that’s right. iAcquire uses the data when creating client reports as it not only helps them to inform the client about how their pages are doing but to also show the importance of certain pages on their site. The data is both a research tool and an education tool.
“Below is a screenshot from a ranking research report showing data we gathered for the keyword ‘inbound marketing tips.’ Moz stats are represented throughout the stats columns. As we work with these reports we are able to see if any of our content distribution efforts resulted in links on page or domain as can be seen in the far left columns.”
That’s how Mozscape is being used today, but it’s only the tip of the iceberg. A few folks have realized the potential outside of the traditional use cases that I’ve mentioned above. The power of the data comes when we take Mozscape data outside of its traditional context of pure link evaluation. Let me show you what I mean.
Its relatively easy to imagine Mozscape’s data being used for link building. With Mozscape’s massive amount of link data, SEOs are able to prioritize their link building efforts, and focus on value added efforts.
You could imagine that some of the examples noted above have been used for link building, but what about a deeper integration into a contact manager? Something that would allow the user to prioritize outreach by the value of a domain.
Just as one can do with the Klout score (or Social Authority) on Twitter, the same can be done for customer relationship efforts in filtering Domain Authority to determine importance.
We’ve seen hints of blogs using Mozscape data determine a top startup list, like the GeekWire 200, but the same could be applied for any rankings list of web properties.
Traditionally, lists have used Alexa or Compete traffic data to determine web prominence, but they’re so inaccurate. Other lists have used social specific metrics like social followings, but those too fall short. Geekwire’s list of the top 200 startups in Seattle uses a blend of both social and web data (External links, MozTrust) to determine just how influential a site is, providing the full picture.
I’m sure we’ve missed a ton of ideas, so we’re calling on you to help us find those new opportunities for Mozscape. Things like a tightening relationship between links and social networks, and categorizing link sources. How would you use this data, and how would you build it? Better yet, why not create your key and get going?
We want to make it easy for you.
We’ve been working quite hard to make our indexes faster and have recently updated our Mozscape API documentation. We want to make it as simple for you to use the data to get your idea up an running as possible.
Plus, if you create something, it’s likely we’ll get you added to our app gallery. We have everything from large corporations to individuals who have used the API and we show off their work in the gallery.
We’d love to hear from you. Obviously we always encourage folks to jump in and check out the free API (as well as the paid), and use the data for something useful for you. We’re also quite open to hearing about ways we can improve our own tools with the data or help educate people better. I look forward to reading through your feedback and seeing if there are ways we can help get people started using Mozscape.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!
Posted by timresnik
Last week, I held a Mozinar outlining a method to extract SERP vertical — called Universal Search by Google — from Google referral strings. Since the Mozinar concluded, the number of people who have reached out with their own theories and ideas has been impressive. I want to post everything that I know here and then leave it up to you folks in the SEOmoz community to start hacking and sharing your insight.
For those of you who did not see the Mozinar, you can access it here (voiceover included). You can also download or view the slides without VO on Slideshare here.
Before getting into the step-by-step process and providing examples of how to use the Google referral string to interpret where in Universal Search your traffic came from, I want to lay out a problem we were having at AudienceWise. In 2011, Matthew Brown and I started an agency to help news publishers with technical SEO and audience development. In our other jobs, specifically Matthew at the New York Times, we struggled with reconciling for the lack of data around Universal Search referrals. As far as our web analytics platforms were concerned, a visit from web search, a News OneBox link, and an image result were all treated exactly the same: as organic search traffic.
Then came Google Secure Search, and referral data got even more opaque. In addition to not knowing which Universal vertical the referral came from, now in about 10% of cases we didn’t even know the keyword that referred the traffic. The question that kept going through our collective ginger minds was: how can we help our clients with content strategy if we know nothing about WHY they are receiving said search traffic? Unfortunately, Secure Search has vastly expanded and now accounts for a large percentage of all Google referral traffic. As way of an example, here is the latest percentage of keyword = (not provided) for SEOmoz:
Matthew and I knew the only way to reclaim *some* of this lost data was to start looking at other sources. Luckily, Matt speaks Spanish (sort of) and came across this blog. The author posited that the ‘ved’ parameter in the Google referral string held some magic in determining the vertical that result appeared in. After doing some quick searches, and looking at the “href” values for the results, it seemed like he was onto something. We immediately set up Google Analytics profile filters to extract this parameter on a client that receives 300,000 search referrals from Google per day. After a couple of hours, we were loaded with enough data to start confirming some of the authors theories and coming up with a few of our own. I will layout what we found, provide a step-by-step tutorial to setup Google Analytics filters, and provide a few examples of how to use the data.
First, let’s talk about where you can find this parameter.
Simply, the Google referral string is the “href” value assigned to each URL in a set of search results. When a user clicks on the above, she is being redirected through a google URL prior to reaching her final destination; Radiohead.com, in this case. Google most likely does this for internal data aggregation reasons — we’re not suppose to know where our traffic comes from, but they sure make use of it — probably for aggregating data around SERPs.
There are two parameters that I will focus on here: ‘cd’ and ‘ved.’ The ‘cd’ parameter has been written about before and tells us the position of the search result in the set. As far as I can tell, the ‘ved’ parameter is divided into three parts and tells us which Universal vertical the result is part of, the position within that vertical (relative position), and the position within the search result (absolute position). I will focus on just the Universal aspect for this post and will follow up with relative vs. absolute position in a follow-up.
Let’s have a look at a few examples.
When QFj is in the ‘ved’ parameter that the result is a standard web search result, such as:
One of the attendees of the Mozinar made this astute observation about a special variation for the web search ‘ved’:
When QqQIw (that’s a capital “i” not a lowercase “L”) it is a Universal result that resides within the Google News OneBox. When QpwI is present that means the result was the thumbnail image within the News OneBox.
You get the idea. Here are some other values of ‘ved.’ I suspect that there are many more and am curious to see what the community here can find and SHARE here within:
You should have a good understanding now of potential power of this information. Did I mention that it is still available even if the keyword is “(not provided)”? We could potentially interpret the keyword by comparing ‘ved.’ Anyone up for the challenge? I go through one example below. While ‘ved’ appears to persist through Secure Search only about 50% of the search referrals within GA have this data. If anyone can shine light on this, I’m sure the rest of the community would shower you with thumbs ups!
Go to the account’s administrative dashboard and select “New Profile.” I would recommend against setting this filter up on an existing profile as that it will overwrite some data that you otherwise want. I called mine ‘Universal Search.’
Next, you will need to set up two advanced filters; one to extract ‘ved’ and ‘cd’ from the Google referral string, and the other to display the data within Google Analytics.
Universal Extract
Here’s the text of the regex that I used
Field A (\?|&)(ved)=([^&]*)
Field B (\?|&)(cd)=([^&]*)
Universal Display
There’s many different ways to do this. I’ve decided to overwrite the campaign dimension of source since that’s where I am checking my organic search referrals.
Filters work while the data is streaming in and will not be reflected retroactively. That’s fine; you just have to wait for a day or so (or an hour or so for bigger sites) to start digging in. Here’s what it should look like:
I prefer to do this level of analysis in Excel, but Advanced Segments can be created to make it all look pretty in GA. I will walk you through the setup of one, which will inform you how to do the rest.
You will want to name your Advanced Segment something that will clue you in to which vertical you are analyzing. In this case, I have called out that it is a standard ‘blue link’ result from a News OneBox. From there, all you need to do is search on ‘Source’ for anything containing the ‘ved’ you are trying to isolate. In this case, we are looking for ‘QqQIw.’
Here’s an example of what you will see:
Wow! There is an actionable result right in front of me. It’s probably time to do some image optimization. Google apparently respects the site as a news authority, but not one that creates good images.
Another useful ‘ved’ to investigate is Sitelinks. Sitelinks are a subset of results triggered by a branded search. Google algorithmically determines which links to include, but webmasters have the ability to demote links in Webmaster Tools. The ‘ved’ parameter can come in handy to measure performance of Sitelink pages and action can be taken. In order to figure out the Sitelink that sent the search referral, look at the ‘cd’ value that was passed with the referral string. We accounted for this in the filters and it is in your data here:
Here’s what the ‘cd’ values mean in relation to Sitelink results:
There are myriad of use cases for bubbling up SEO action items. Here are a few, and please add more in the comments:
Events Markup: ved = xxxBE0MGM
Music Markup: ved = xxxQ6hEw
That about wraps it up for my first — of hopefully many — posts on ‘ved.’ In the months to come, Moz will be collecting Google referral string data on a great number of SERPs for various keywords. We plan to unleash our data hound to sniff out the most useful elements. In the meantime, I would like to use this post as a place for the hacking to begin and the sharing of your thoughts in the comments.
Dig in!
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!
Posted by timresnik
Last week, I held a Mozinar outlining a method to extract SERP vertical — called Universal Search by Google — from Google referral strings. Since the Mozinar concluded, the number of people who have reached out with their own theories and ideas has been impressive. I want to post everything that I know here and then leave it up to you folks in the SEOmoz community to start hacking and sharing your insight.
For those of you who did not see the Mozinar, you can access it here (voiceover included). You can also download or view the slides without VO on Slideshare here.
Before getting into the step-by-step process and providing examples of how to use the Google referral string to interpret where in Universal Search your traffic came from, I want to lay out a problem we were having at AudienceWise. In 2011, Matthew Brown and I started an agency to help news publishers with technical SEO and audience development. In our other jobs, specifically Matthew at the New York Times, we struggled with reconciling for the lack of data around Universal Search referrals. As far as our web analytics platforms were concerned, a visit from web search, a News OneBox link, and an image result were all treated exactly the same: as organic search traffic.
Then came Google Secure Search, and referral data got even more opaque. In addition to not knowing which Universal vertical the referral came from, now in about 10% of cases we didn’t even know the keyword that referred the traffic. The question that kept going through our collective ginger minds was: how can we help our clients with content strategy if we know nothing about WHY they are receiving said search traffic? Unfortunately, Secure Search has vastly expanded and now accounts for a large percentage of all Google referral traffic. As way of an example, here is the latest percentage of keyword = (not provided) for SEOmoz:
Matthew and I knew the only way to reclaim *some* of this lost data was to start looking at other sources. Luckily, Matt speaks Spanish (sort of) and came across this blog. The author posited that the ‘ved’ parameter in the Google referral string held some magic in determining the vertical that result appeared in. After doing some quick searches, and looking at the “href” values for the results, it seemed like he was onto something. We immediately set up Google Analytics profile filters to extract this parameter on a client that receives 300,000 search referrals from Google per day. After a couple of hours, we were loaded with enough data to start confirming some of the authors theories and coming up with a few of our own. I will layout what we found, provide a step-by-step tutorial to setup Google Analytics filters, and provide a few examples of how to use the data.
First, let’s talk about where you can find this parameter.
Simply, the Google referral string is the “href” value assigned to each URL in a set of search results. When a user clicks on the above, she is being redirected through a google URL prior to reaching her final destination; Radiohead.com, in this case. Google most likely does this for internal data aggregation reasons — we’re not suppose to know where our traffic comes from, but they sure make use of it — probably for aggregating data around SERPs.
There are two parameters that I will focus on here: ‘cd’ and ‘ved.’ The ‘cd’ parameter has been written about before and tells us the position of the search result in the set. As far as I can tell, the ‘ved’ parameter is divided into three parts and tells us which Universal vertical the result is part of, the position within that vertical (relative position), and the position within the search result (absolute position). I will focus on just the Universal aspect for this post and will follow up with relative vs. absolute position in a follow-up.
Let’s have a look at a few examples.
When QFj is in the ‘ved’ parameter that the result is a standard web search result, such as:
One of the attendees of the Mozinar made this astute observation about a special variation for the web search ‘ved’:
When QqQIw (that’s a capital “i” not a lowercase “L”) it is a Universal result that resides within the Google News OneBox. When QpwI is present that means the result was the thumbnail image within the News OneBox.
You get the idea. Here are some other values of ‘ved.’ I suspect that there are many more and am curious to see what the community here can find and SHARE here within:
You should have a good understanding now of potential power of this information. Did I mention that it is still available even if the keyword is “(not provided)”? We could potentially interpret the keyword by comparing ‘ved.’ Anyone up for the challenge? I go through one example below. While ‘ved’ appears to persist through Secure Search only about 50% of the search referrals within GA have this data. If anyone can shine light on this, I’m sure the rest of the community would shower you with thumbs ups!
Go to the account’s administrative dashboard and select “New Profile.” I would recommend against setting this filter up on an existing profile as that it will overwrite some data that you otherwise want. I called mine ‘Universal Search.’
Next, you will need to set up two advanced filters; one to extract ‘ved’ and ‘cd’ from the Google referral string, and the other to display the data within Google Analytics.
Universal Extract
Here’s the text of the regex that I used
Field A (\?|&)(ved)=([^&]*)
Field B (\?|&)(cd)=([^&]*)
Universal Display
There’s many different ways to do this. I’ve decided to overwrite the campaign dimension of source since that’s where I am checking my organic search referrals.
Filters work while the data is streaming in and will not be reflected retroactively. That’s fine; you just have to wait for a day or so (or an hour or so for bigger sites) to start digging in. Here’s what it should look like:
I prefer to do this level of analysis in Excel, but Advanced Segments can be created to make it all look pretty in GA. I will walk you through the setup of one, which will inform you how to do the rest.
You will want to name your Advanced Segment something that will clue you in to which vertical you are analyzing. In this case, I have called out that it is a standard ‘blue link’ result from a News OneBox. From there, all you need to do is search on ‘Source’ for anything containing the ‘ved’ you are trying to isolate. In this case, we are looking for ‘QqQIw.’
Here’s an example of what you will see:
Wow! There is an actionable result right in front of me. It’s probably time to do some image optimization. Google apparently respects the site as a news authority, but not one that creates good images.
Another useful ‘ved’ to investigate is Sitelinks. Sitelinks are a subset of results triggered by a branded search. Google algorithmically determines which links to include, but webmasters have the ability to demote links in Webmaster Tools. The ‘ved’ parameter can come in handy to measure performance of Sitelink pages and action can be taken. In order to figure out the Sitelink that sent the search referral, look at the ‘cd’ value that was passed with the referral string. We accounted for this in the filters and it is in your data here:
Here’s what the ‘cd’ values mean in relation to Sitelink results:
There are myriad of use cases for bubbling up SEO action items. Here are a few, and please add more in the comments:
Events Markup: ved = xxxBE0MGM
Music Markup: ved = xxxQ6hEw
That about wraps it up for my first — of hopefully many — posts on ‘ved.’ In the months to come, Moz will be collecting Google referral string data on a great number of SERPs for various keywords. We plan to unleash our data hound to sniff out the most useful elements. In the meantime, I would like to use this post as a place for the hacking to begin and the sharing of your thoughts in the comments.
Dig in!
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!