Archives

 

Google Changes Referrer Values Again For Secure Searches

Over the past 6 months Google has made changes to their search experience in an attempt to increase the privacy and security of their signed in users. What this has meant for analytics tools is that the referring URL for those signed in users was stripped of any searched keywords when clicking on Google organic search results.

Here’s what has been happening behind the scenes. All signed in users are now on a secure version of Google (https), and a redirect has been added to each search results click. That redirect is to a non-secure page (http), where the referring URL is changed before the visitor arrives at the page they requested. That new referring URL value has had its keywords removed, but still contains enough information to determine it was a Google Secure search. Workarounds were created to help identify a Google Secure search in SiteCatalyst keyword reporting, as well as Omniture making a change to try and account for those searches.

Since making that change Google has determined that the additional redirect is unnecessary and potentially slowed down the users experience, so they have decided to eliminate it (unfortunately that does not mean analytics tools will be able to see those keywords again).

Today Google announced a change to the way they plan on handling referring URL’s starting in April 2012. Google has decided that they will now begin to use the referrer meta tag for browsers that will support it, as opposed to the redirect to the non secure page. Currently the only major browser that supports it is Google Chrome.

If you are not familiar with the referrer meta tag, what it does is it lets each web page decide how referrer from it should be handled. For example, here’s what a meta referrer tag looks like:

<meta name="referrer" content="never">

What this tag will do is that it tells the browser to never pass any referring information from the page its on. The browser should then set the referrer header value to a blank string for referrers from that page.

Fortunately Google is not going to that extreme. They have decided to use the “origin” value:

<meta name="referrer" content="origin">

This is the referrer meta tag value that Google will begin to use in April 2012. When the change goes live, all search clicks from signed in users will now only have the referrer value of https://www.google.com/. There will be no other information in the referring URL, so no way to determine that it was specifically a Google secure search other than the URL being simply that host value. Non-secure searches, ones made from a user not logged into a Google account, will continue to function in the same way as they do now.

Currently the referrer meta tag is not currently supported in all browsers. I tried it using Chrome 17 and it is working. Testing it in Safari 5.1.4 and Firefox 11, the referrer meta tag has no impact.

So what does this mean as far as SiteCatalyst reporting? According to the Knowledge Base answer #5329, “If the domain of the referrer corresponds to that of a recognized search engine (e.g. “google.com”) and contains the recognized search keyword query parameter for the given search engine domain (e.g. for Google this is “q=”), then the referrer is considered a search engine, and the value of the keyword query parameter is taken as the search keyword.” So no search keyword query parameter, no search is counted. Currently for a Google secure search, the parameter is still there, but it’s unpopulated. Now Google plans to remove it all together.

Hopefully before Google rolls this out publicly Adobe will come up with a solution for SiteCatalyst so there are no interruptions in the Search Engine reporting. If Adobe does not get to it in time, or if Google decides to push the change out before April, then a couple of lines of code added to your s_code.js file will keep the impact to a minimum while Adobe works out a solution.

if(document.referrer=="https://www.google.com/"){
	s.referrer="https://www.google.com/?q=google%20secure%20search";
}

What this will do is look for a referrer with the exact value of https://www.google.com/ and append a q= value to it with the keyword of google secure search. If the .com version of Google is not the one most used by your visitors, then just replace it with the correct tld version applicable to your site. This snippet of code will make sure the search is still counted, and you will continue to keep the same level of reporting that you have now.

UPDATE: If your your visitors are coming to your site from multiple country specific versions of Google, then I have you covered. Just include this plugin in your s_code file and all Google secure searches from every Google domain will automatically be handled. All you need to do is cut and paste.

s.getGoogle=new Function(""
+"var s=this,a=document.referrer,b=a.split('/')[3],c=a.substring(0,19"
+");a&&26>=a.length&&(b||c=='https://www.google.'&&(this.s.referrer=a"
+"+'?q=google%20secure%20search'));return this.s.referrer")();

enjoy!

Improve Accuracy & Identify Traffic That SiteCatalyst Can’t

I’ve been doing a lot of work recently with my Traffic Sources reports. My goals have been to clean up messy data that could come in, and to make it easier to look at traffic from different sections of the same referrer. Now I would like to see what I can do to make the standard Referrer and Referring Domains reports a little more accurate, and try to fill in some of the holes they create which prevent me from getting a really good summary view of my traffic.

Overall the standard Referrer and Referring Domains reports do a pretty good job at telling me where my visitors came from, but there is one item that is a major problem for me. That one item is called “Typed/Bookmarked”.

According to the SiteCatalyst Knowledge Base, “Typed/Bookmarked line items occur in reporting where a referrer for an image request is not present.” So in other words, if SiteCatalyst does not see a referrer value, then it simply can not tell you where that visitor came from, so they get dropped into the Typed/Bookmarked bucket. Typically that’s fine. There is no way to know completely where every single one of your visitors came from. Thats just the nature of the beast. But one problem I have is that even though SiteCatalyst may not know where that visitor came from, I possibly do. So how do I know but SiteCatalyst doesn’t you ask? Tracking codes.

Yesterday Omniture shared a study that was done by BtoB Magazine which said “email is used by 88% of marketers surveyed and ranked as their No. 1 form of digital outreach”. Its been no secret that running email campaigns is a great way to get more visits for your site, and ultimately more conversions. Judging by my inbox there are a lot of marketers out there that agree. Email marketing best practices recommend that tracking codes are included on all of the URL’s in the email. This is typically the best way to determine the effectiveness of those email campaigns, and hopefully it’s something you’ve been doing with your own email campaigns. The problem with this is that most email applications are not going to pass a referrer value to the site. So even though we are able to track the performance of these emails in our campaign reporting, when looking at our Referrer reports we are not able to see that traffic credited to the correct source. No referring value means the Traffic Sources reports consider it to be Typed/Bookmarked traffic, when we know it isn’t. Our Typed/Bookmarked values get over inflated, and the email campaign traffic doesn’t get properly credited. So what can be done about it?

Here’s what I like to do. I add in a tiny bit of code to the doPlugins section of my s_code.js file that checks to see if the image request has no referring URL, and if the current URL has a tracking code associated with one of my email campaigns. If that criteria is met then inject a specific referring domain value to my traffic sources report, correctly attributing that visit as being from one of my email campaigns. The code to do this is quite simple:

if(!document.referrer){
	var s_eml = s.getQueryParam('eml');
	if(s_eml){s.referrer="mail://email.campaign/"+s_eml;}
}

Now lets say I’m running an email campaign which contains links to my home page. I made sure that the URL’s for those links have the query string parameter eml=56789. The parameter eml is the tracking code I use specifically for my email campaigns, and 56789 is the identifier for that specific campaign. When a visitor tries to access a page of my site using one of those URL’s containing my email tracking code and they do not have a referring URL value, my normal campaign tracking does it’s job, and this new snippet of code inject’s the value of mail://email.campaign/56789 as the referring domain. If the visitor was using using an email application that did pass a referrer value, then that passed value will always take precedence. Injecting that new value as the referring URL will accomplish a couple of things. First that whole value will now appear in my Referrers report. With that I’m able to compare the traffic generated from specific email campaigns to other traffic sources. Comparing traffic generated from an email campaign to traffic generated from an organic source wasn’t always the easiest thing to do in a single SiteCatalyst report.
Email Referrer

Next in my Referring Domains report I will get the value of email.campaign, and more importantly I won’t register another instance of Typed/Bookmarked. With this I can get a look at the traffic generated from the email campaign compared to all my other known referrers as a whole to see how it stacks up.
Email Referring Domains

Here’s an additional bonus I get from doing this. If you take a look at that URL value I used as the referrer value, it does not begin with http://, but it begins with mail://. In the Traffic Sources reports you will find a report called Referrer Type. This report is basically a glorified SAINT classification that looks at each referring URL and assigns it to a different bucket. When a SiteCatalyst see’s a referring URL beginning with the value of mail:// or the value of imap:// it then gets classified to the Mail bucket in the Referrer Type report. I’m now starting to get a better view of all my traffic sources in one pretty graph.

Email Referrer Types

Another source of traffic for some sites that is also not being accurately represented in the Traffic Sources reports is when a visitor comes to the site by clicking a link in a pdf. Last week after the latest iPad was announced, a friend sent me a pdf that came from Apple about using the iPad at Work. It contained good information, so I passed it along to a couple of other friends. Looking at the pdf a little closer, there was one thing that caught my eye. It contained 25 individual links back to apple.com. I wasn’t viewing this email in a web browser but in the simple Preview app on my Mac, yet every single one of those links was clickable. I thought that was great, another opportunities to drive traffic to the site. But this was also not so great because it was another opportunities to take a visitor and classify them as being Typed/Bookmarked, even though Apple could know easily and specifically where they were coming from.

Much like with emails, all it takes is a tiny couple of lines of code to identify that traffic. I like to use the value of pdf= as the query string parameter for links embedded in pdf’s, but you can obviously change it to whatever you like.

if(!document.referrer){
	var s_pdf = s.getQueryParam('pdf');
	if(s_pdf){s.referrer="file://pdf.document/"+s_pdf;}
}

Just like before with the emails, the current URL needs to contain that tracking code and there must be no referrer value present for this to work.

Taking a look at that snippet of code, I used the referring URL domain value of file://pdf.document combined with a unique identifier for that pdf. Unlike with the email’s, this time I started the value with file://. This will now get assigned a new value in the Referrer Type report, the value of Hard Drive. Not the best description of what’s going on, but that’s the only value left. The Referrer Type report is set to classify all referrers into 6 different buckets, and there is no way to add any additional categories to it.

All Referrer Types

Another popular source of traffic I’ve seen that does not get proper credit in Traffic Sources reports are visits that come from a mobile application. There are tons of mobile apps which have some component to them that can take the visitor from the app to their website. Like all the other links, you hopefully have the ones in your mobile app tagged with a tracking code. If so then just one more snippet of code and that traffic can be accounted for:

if(!document.referrer){
	var s_app = s.getQueryParam('app');
	if(s_app){s.referrer="file://mobile.application/"+s_app;}
}

I used the tracking code of app, but you can use what ever you would like. This app referred traffic will also be listed in the referrer type report as Hard Drive. Since I used up all the options in that report, all the extras such as mobile app traffic and pdf traffic will need to share the Hard Drive bucket.

I like to keep all three snippets of code wrapped up in one clean package:

if(!document.referrer){
	var s_eml = s.getQueryParam('eml');
	var s_pdf = s.getQueryParam('pdf');
	var s_app = s.getQueryParam('app');

	if(s_eml){
		s.referrer = "mail://email.campaign/"+s_eml;
	}else if(s_pdf){
		s.referrer = "file:///pdf.document/"+s_pdf;
	}else if(s_app){
		s.referrer = "file:///mobile.application/"+s_app;
	}
}

Now in my Referrer and Referring Domains reporting, I get a better look at how my visitors are arriving to my site.
All New Referrers

All New Referring Domains

Maybe you have some other kind of web app, or some kind of shared widget, or some other totally different way for visitors to follow a link to get to your site. So long as you can add a tracking code to that link you can get that traffic correctly represented in your Traffic Sources reports, and stop over inflating the Typed/Bookmarked metric. This is not meant to replace the Marketing Channels reports, the Channel Manager plugin, or the Unified Sources VISTA rule, but to improve the functionality and accuracy of some of the core SiteCatalyst reports.

I hope this helps.
enjoy!

How To Fix Google Plus Referrer Values In SiteCatalyst

Recently I was looking through some traffic sources reports in SiteCatalyst, and I noticed something odd. In my Referrers report I had a bunch of different referring URL’s that were all coming from Google Plus, all going to the exact same page on my site, but each referring URL was listed separately. Taking a closer look, I noticed that each referring URL contained some unique query string values, and my Referrers report was quickly getting filled up with a lot of redundant links.

Multiple Google Plus Referrers

I was curious to learn about what was going on, so using Chrome I logged into my Google Plus account looking for a link to click. I found a link one of my friends shared, clicked it, and then checked what the referring link to that site was.
Google Plus Referrer using Chrome
Next I did the same thing in Firefox. I clicked the exact same link as before shared by the same person, and then checked that referrer.
Google Plus Referrer using Firefox
Finally I tried clicking the exact same link for a third time, this time trying it in Safari.
Google Plus Referrer using Safari
The exact same link, shared by the exact same person, going to the exact same page, from the exact same Google Plus account, but three different referring URL values. It looked like Google Plus was using a unique URL value for each browser. But wait, I had a ton of different values in my Referrers report so that couldn’t be it. Still using Chrome I went back and clicked on the same link I’ve been using.
Google Plus Referrer using Chrome again
Another unique referring URL. As Google Plus usage beings to increase, I can see this becoming a big problem. I needed to find an easy way to clean up my Google Plus referrers.

Looking at each referring link I see that there are two parameters that are unique each time, n= and usg=. I could probably keep testing and find out what these two parameters meant, but I doubt I’ll need them. I just need to know that the visitor came from Google Plus. But looking closer at the referrer value I can see something else that could be useful. The url= parameter contains the page that the link was headed to, and that could be helpful to have. So if I had a way to just drop off everything from that Google Plus referring URL query string except for the url= value, that would give me what I’m looking for. I would now be able to clean up my Referrers report by removing all the redundant links, and as a bonus I would be able to tell which page of my site attracted the most attention on Google Plus just by looking at those referrers. So how do I do it?

All it takes is a little bit of code dropped into the s_code.js file and my problem is solved.
Google Plus Referrer's Cleaned Up In SiteCatalyst

The best part is that there is no set up required. I wanted to keep it really simple so I made sure that there’s nothing that needs to be configured. Just paste this plugin code right next to all of your other plugins, and all of your messy Google Plus referring URLs will get cleaned up automagically!

/*
 * Clean up Google Plus referrer values
 */
s.cleanGP=new Function("var s=this,a=document.referrer,b='plus.url.go"
+"ogle.com',c,d,e,g,i=0,p,referrer;if(a&&a.split('/')[2]==b){c=a.spli"
+"t(/[\?|&]/);d=c[0].length;e=c[0].lastIndexOf('/');g=d>e?c[0].substr"
+"ing(0,e+1):c[0];for(i=0;i<=c.length;i++)'url'==c[i].substring(0,3)&"
+"&(p='?'+unescape(c[i]),i=c.length+1);g&&(this.s.referrer=p?g+p:g);}"
+"return this.s.referrer")();

enjoy!