Improve Accuracy & Identify Traffic That SiteCatalyst Can’t

I’ve been doing a lot of work recently with my Traffic Sources reports. My goals have been to clean up messy data that could come in, and to make it easier to look at traffic from different sections of the same referrer. Now I would like to see what I can do to make the standard Referrer and Referring Domains reports a little more accurate, and try to fill in some of the holes they create which prevent me from getting a really good summary view of my traffic.

Overall the standard Referrer and Referring Domains reports do a pretty good job at telling me where my visitors came from, but there is one item that is a major problem for me. That one item is called “Typed/Bookmarked”.

According to the SiteCatalyst Knowledge Base, “Typed/Bookmarked line items occur in reporting where a referrer for an image request is not present.” So in other words, if SiteCatalyst does not see a referrer value, then it simply can not tell you where that visitor came from, so they get dropped into the Typed/Bookmarked bucket. Typically that’s fine. There is no way to know completely where every single one of your visitors came from. Thats just the nature of the beast. But one problem I have is that even though SiteCatalyst may not know where that visitor came from, I possibly do. So how do I know but SiteCatalyst doesn’t you ask? Tracking codes.

Yesterday Omniture shared a study that was done by BtoB Magazine which said “email is used by 88% of marketers surveyed and ranked as their No. 1 form of digital outreach”. Its been no secret that running email campaigns is a great way to get more visits for your site, and ultimately more conversions. Judging by my inbox there are a lot of marketers out there that agree. Email marketing best practices recommend that tracking codes are included on all of the URL’s in the email. This is typically the best way to determine the effectiveness of those email campaigns, and hopefully it’s something you’ve been doing with your own email campaigns. The problem with this is that most email applications are not going to pass a referrer value to the site. So even though we are able to track the performance of these emails in our campaign reporting, when looking at our Referrer reports we are not able to see that traffic credited to the correct source. No referring value means the Traffic Sources reports consider it to be Typed/Bookmarked traffic, when we know it isn’t. Our Typed/Bookmarked values get over inflated, and the email campaign traffic doesn’t get properly credited. So what can be done about it?

Here’s what I like to do. I add in a tiny bit of code to the doPlugins section of my s_code.js file that checks to see if the image request has no referring URL, and if the current URL has a tracking code associated with one of my email campaigns. If that criteria is met then inject a specific referring domain value to my traffic sources report, correctly attributing that visit as being from one of my email campaigns. The code to do this is quite simple:

if(!document.referrer){
	var s_eml = s.getQueryParam('eml');
	if(s_eml){s.referrer="mail://email.campaign/"+s_eml;}
}

Now lets say I’m running an email campaign which contains links to my home page. I made sure that the URL’s for those links have the query string parameter eml=56789. The parameter eml is the tracking code I use specifically for my email campaigns, and 56789 is the identifier for that specific campaign. When a visitor tries to access a page of my site using one of those URL’s containing my email tracking code and they do not have a referring URL value, my normal campaign tracking does it’s job, and this new snippet of code inject’s the value of mail://email.campaign/56789 as the referring domain. If the visitor was using using an email application that did pass a referrer value, then that passed value will always take precedence. Injecting that new value as the referring URL will accomplish a couple of things. First that whole value will now appear in my Referrers report. With that I’m able to compare the traffic generated from specific email campaigns to other traffic sources. Comparing traffic generated from an email campaign to traffic generated from an organic source wasn’t always the easiest thing to do in a single SiteCatalyst report.
Email Referrer

Next in my Referring Domains report I will get the value of email.campaign, and more importantly I won’t register another instance of Typed/Bookmarked. With this I can get a look at the traffic generated from the email campaign compared to all my other known referrers as a whole to see how it stacks up.
Email Referring Domains

Here’s an additional bonus I get from doing this. If you take a look at that URL value I used as the referrer value, it does not begin with http://, but it begins with mail://. In the Traffic Sources reports you will find a report called Referrer Type. This report is basically a glorified SAINT classification that looks at each referring URL and assigns it to a different bucket. When a SiteCatalyst see’s a referring URL beginning with the value of mail:// or the value of imap:// it then gets classified to the Mail bucket in the Referrer Type report. I’m now starting to get a better view of all my traffic sources in one pretty graph.

Email Referrer Types

Another source of traffic for some sites that is also not being accurately represented in the Traffic Sources reports is when a visitor comes to the site by clicking a link in a pdf. Last week after the latest iPad was announced, a friend sent me a pdf that came from Apple about using the iPad at Work. It contained good information, so I passed it along to a couple of other friends. Looking at the pdf a little closer, there was one thing that caught my eye. It contained 25 individual links back to apple.com. I wasn’t viewing this email in a web browser but in the simple Preview app on my Mac, yet every single one of those links was clickable. I thought that was great, another opportunities to drive traffic to the site. But this was also not so great because it was another opportunities to take a visitor and classify them as being Typed/Bookmarked, even though Apple could know easily and specifically where they were coming from.

Much like with emails, all it takes is a tiny couple of lines of code to identify that traffic. I like to use the value of pdf= as the query string parameter for links embedded in pdf’s, but you can obviously change it to whatever you like.

if(!document.referrer){
	var s_pdf = s.getQueryParam('pdf');
	if(s_pdf){s.referrer="file://pdf.document/"+s_pdf;}
}

Just like before with the emails, the current URL needs to contain that tracking code and there must be no referrer value present for this to work.

Taking a look at that snippet of code, I used the referring URL domain value of file://pdf.document combined with a unique identifier for that pdf. Unlike with the email’s, this time I started the value with file://. This will now get assigned a new value in the Referrer Type report, the value of Hard Drive. Not the best description of what’s going on, but that’s the only value left. The Referrer Type report is set to classify all referrers into 6 different buckets, and there is no way to add any additional categories to it.

All Referrer Types

Another popular source of traffic I’ve seen that does not get proper credit in Traffic Sources reports are visits that come from a mobile application. There are tons of mobile apps which have some component to them that can take the visitor from the app to their website. Like all the other links, you hopefully have the ones in your mobile app tagged with a tracking code. If so then just one more snippet of code and that traffic can be accounted for:

if(!document.referrer){
	var s_app = s.getQueryParam('app');
	if(s_app){s.referrer="file://mobile.application/"+s_app;}
}

I used the tracking code of app, but you can use what ever you would like. This app referred traffic will also be listed in the referrer type report as Hard Drive. Since I used up all the options in that report, all the extras such as mobile app traffic and pdf traffic will need to share the Hard Drive bucket.

I like to keep all three snippets of code wrapped up in one clean package:

if(!document.referrer){
	var s_eml = s.getQueryParam('eml');
	var s_pdf = s.getQueryParam('pdf');
	var s_app = s.getQueryParam('app');

	if(s_eml){
		s.referrer = "mail://email.campaign/"+s_eml;
	}else if(s_pdf){
		s.referrer = "file:///pdf.document/"+s_pdf;
	}else if(s_app){
		s.referrer = "file:///mobile.application/"+s_app;
	}
}

Now in my Referrer and Referring Domains reporting, I get a better look at how my visitors are arriving to my site.
All New Referrers

All New Referring Domains

Maybe you have some other kind of web app, or some kind of shared widget, or some other totally different way for visitors to follow a link to get to your site. So long as you can add a tracking code to that link you can get that traffic correctly represented in your Traffic Sources reports, and stop over inflating the Typed/Bookmarked metric. This is not meant to replace the Marketing Channels reports, the Channel Manager plugin, or the Unified Sources VISTA rule, but to improve the functionality and accuracy of some of the core SiteCatalyst reports.

I hope this helps.
enjoy!

Comments:

7 Responses to “Improve Accuracy & Identify Traffic That SiteCatalyst Can’t”


  1. Very nice writeup; thanks for posting!


  2. Simple and cool!


  3. […] Rogers was and so he decided to do something about it. And then conveniently he wrote a blog post about it as well, so that everyone else can find out what he did about it. There is no way to know completely where […]


  4. Hey Kevin,

    Great read. I asked you a question earlier on Twitter. What kind of programming/technique do you use to code the s_code.js file? If someone who doesn’t have access to do that kind of modification, is there anywhere else to look in SiteCatalyst to better determine bookmarked/typed source?

    Thank you!


  5. All of the coding is done using JavaScript.

    I think a better explanation of the “Typed/Bookmarked” traffic would be to say that we don’t know where they came from. Typically when a visitor comes to your site you will have the value of the URL that they just came from in the document.referrer JavaScript variable. That is whats used to power all of the Referrer reports in SiteCatalyst. When a visitor comes to your site and that variable is blank, then SiteCatalyst has no way to know where they came from so all of that traffic gets dumped into the “we really don’t know where these guys came from” bucket, that has the friendly name of “Typed/Bookmarked”. Typically when a visitor’s referrer value can’t be determined then it is pretty safe to say that they either typed in your URL directly, or they had your site bookmarked in their browser. Thats why putting all that traffic on the “Typed/Bookmarked” bucket is a pretty safe bet.


  6. Great post. I was hoping if you could help me solve a problem (or instruct me how to write a s_code.js code)

    A vendor managing one of our client’s site (And I have no access to that site nor the site itself has Omniture tracking implemented) proclaims they are sending traffic to the main site (which I have access to and the main site has Omniture tracking). When I review the referring domain report, I do not see traffic coming from that vendor site. I believe the variable on that site is missing.

    By the way, that site our vendor manages is a secured (https://) one and only has Google Analytics tracking. Is there any way I can either write a code and ask them to implement so that I can see referring domains in my Omniture suite? Thanks!


  7. The first thing I would check is if the client site is flagged as an internal domain. To check for this, go to the Report Suite Manager in the SiteCatalyst Admin Console. In the list of your report suites, select the report suite in question, then go to Edit Settings > General > Internal URL Filters. If any referring domain matches one of the items listed in there then it will not appear in the Referring Domains report. One thing to remember is that there does not need to be a full exact match for a referrer to get flagged as being internal. For example if you have “example.com” listed in the Internal URL Filters, it will flag traffic from http://www.example.com, dev.example.com, and whatever.example.com as being internal. And that is typically what we would want, but it will also flag traffic from best-example.com, newexample.com, and whateverexample.com as being internal and those will not appear in any traffic reports. Also make sure you don’t have a single period listed as one of the filters. That will prevent any external traffic from being counted. Another thing you may want to do is to get the vendor to tag all of the links on the clients site that lead to the main site with tracking codes. Good luck!

Leave a Reply