A Better Solution to Measure the Real Traffic to Your Blog?

July 22, 2009
By mayanks

The blogosphere is quite excited with the fact that Twitter is directing a lot of traffic to blogs & websites. But no one has been able to conclusively measure that traffic. Fred Wilson, struggled to interpret the data  from Google Analytics and measure exactly how much traffic came from Twitter. Meanwhile Danny Sullivian, ran his own experiment by tweeting a short link containing a tracking code and seeing how much bit.ly reported against Google Analytics report and his own server logs. The results seemed quite good but the experiment itself was not yet perfect. Bit.ly’s link counting is inherently flawed as I’ve explained in my previous blog.

It is somewhat surprising to me that after having Google Analytics for so long all of us are still struggling to measure correctly a specific channel of traffic to our site. As I have been working with several email marketing tools these last fews a simple solution struck me. I was quite struck by the simplicity of the solution and was wondered anyone has thought of it before? I’d urge readers (not you mom) including Danny and Fred try out this little experiment on their blog and see if they can quantify the exact amount of traffic coming in through Twitter (or bit.ly).

The Solution

Nearly all email marketing service providers and products such as AWeber, Campaign Monitor or MailChimp use a simple method to track how many people have opened their mails. Since JavaScript is not allowed inside the mail content, a Google Analytics kind of solution is definitely not possible. What they do is they embed a tiny (1×1 px) transparent image in the mail. When the mail is opened in an email client and the image is loaded, the server gets a GET request for that image and immediately knows that someone has opened the email.

On the same lines, for this blog post I have included an image in this blog post. But instead of directly storing the url of that image, I have first shortened it using bit.ly. When this blog is loaded on your browser, even if JavaScript is disabled, that image will be loaded, and hence bit.ly will register the count. Search bots, Twitter clients and browser plugins are not going to be interested in my image. So clearly the number of views reported by bit.ly for that image (and not for the blog’s permalink which I tweeted) will be a correct estimation of the number of humans who have clicked on the shortened link in Twitter. This will be smaller than what bit.ly would have recorded for the blog itself, and bigger than what Google Analytics will report if this blog is viewed in a phone which has JavaScript disabled.

Conclusion

This blog does not get a lot of traffic (yet), so the numbers may not make good sense. But if high traffic bloggers such as Danny or Fred Wilson use this little experiment on their blog, it would be worth seeing what the results are like.

This should also give Bit.ly some ideas on how to accurately measure link views and discard the requestes for only a link lookup. The solution may be involved as the user would have to embed a shortened link to a tracking image, but it should be ok if the user is really motivatedand wants to measure his traffic. Or better still, similar to Google Analytics, ask users to embed a small piece of code in the blog entry that they want to track. This code can be as generic as follows

<img src="http://bit.ly/mayanks/tracking/image">

When the browser makes a request for this image as part of loading the blog entry, the HTTP_REFERER should identify the permalink of the blog entry for which the request is being made. Just increment the counter for that permalink and you are done. I’d love to hear from others who have tried something along these lines and the results you see if you try this image embedding with a bit.ly link.

Reblog this post [with Zemanta]

Tags: ,

6 Responses to A Better Solution to Measure the Real Traffic to Your Blog?

  1. Kishore on July 23, 2009 at 8:00 am

    Hey Mayank,

    Interesting thought there. It seems like a good way to measure page views for your site. Though it still does not measure how much traffic you got from twitter or from a shortened link, but how much traffic you got in general irrespective of the source.

    > So clearly the number of views reported by bit.ly for that image (and not for the blog’s permalink which I tweeted) will be a correct estimation of the number of humans who have clicked on the shortened link in Twitter

    Is actually the number of users who visited your blog from any source not just twitter.

    Also note that search engines do cache your entire site (including images) and browser plugins like preview plugins will look up the image also.

    I like the solution and believe it will provide a better means right now to measure human traffic.

  2. fred wilson on July 23, 2009 at 11:00 am

    neat idea. i’ll forward this to the bit.ly folks. thanks!

  3. kortina on July 23, 2009 at 6:34 pm

    A tracking image is a classic way to measure traffic to a page, and I would agree that it presents a more accurate view of mobile traffic than Google Analytics. The tracking image, however, is more prone to double counting of browser traffic, so I think the best approach is to use Google Analytics as your primary counter, and then use a bit.ly-ized image for measuring mobile traffic.

    One place where a bit.ly-ized image has a real advantage is when you do not control the page source or do not have access to Google Analytics, but still want to estimate traffic. Here’s a post about how my sister used to track the number of page views of a Craigslist ad, for example: http://bit.ly/3gTktk

  4. Mayank on July 26, 2009 at 12:01 am

    Kortina, that is a real cool hack your sister did on Craiglist. A very nice and innovative way of getting around the problem. You are right in stating that Google Analytics should be used as primary source of statistic. The hack that I am suggesting is primarily to get accurate bit.ly counting disregarding the automatic lookups that browser plugins and some twitter clients do for a shortened link. Bit.ly’s counting becomes important when you want to measure the traffic from Twitter exclusively where you are going to publish your shortened link and would also like to track people who would be viewing the page on their mobile phones which invariably have JavaScript turned off and hence are not measured through Google Analytics.

  5. Mayank on July 26, 2009 at 12:05 am

    Kishore,
    A very important point you raised. I realized about this problem after writing the blog but before publishing it. So the way I solved the problem was to include a tracking code in my blog’s permalink which I then shortened using bit.ly. Then the image’s shortlink count on bit.ly will have largely two different Referers. One directly coming to the blog and containing the blog’s permalink as its referer. and the other set coming by clicking through shortened link containing the modified blog’s permalink. By only taking the second count I’ll know exact number of people who visited the site through twitter.

  6. Bruce on May 21, 2010 at 1:53 pm

    Hey Mayank,

    Interesting thought there. It seems like a good way to measure page views for your site. Though it still does not measure how much traffic you got from twitter or from a shortened link, but how much traffic you got in general irrespective of the source.

    > So clearly the number of views reported by bit.ly for that image (and not for the blog’s permalink which I tweeted) will be a correct estimation of the number of humans who have clicked on the shortened link in Twitter

    Is actually the number of users who visited your blog from any source not just twitter.

    Also note that search engines do cache your entire site (including images) and browser plugins like preview plugins will look up the image also.

    I like the solution and believe it will provide a better means right now to measure human traffic.

Leave a Reply

Your email address will not be published. Required fields are marked *

*