Both Alexa and Google Trends have a beguiling attraction for many. You can put any website address and, unless it’s a very small site, you will get graphs of traffic to the site, options to refine the graphs, numbers with lots of decimal places (in the case of Alexa – and how can numbers with decimal points in them be wrong?) and geographic breakdowns (in the case of Google – and how can lists of places be wrong?).
It all looks very impressive and many people take their results as gospel. But should they?
Alexa has had a checkered history, particularly as its data use to be dependent on users of its toolbar. Although Alexa has long since started getting data from other sources too, it is still hard to tell how good a balance there is. Google is just as secretive about how it does things. But it’s Google and Alexa is now owned by Amazon. Google and Amazon are both firms that seem to know what they’re doing and have good reputations. So perhaps it’s ok to trust them even if we don’t know the whys and the hows?
There are three levels of argument here.
First, there is the naivity displayed by far too many, who simply take Alexa and Google at face value and look rather confused if you question why they trust the data, not having thought about any possible problems with them. I’ve recently been through a dozen and more demonstrations of online monitoring software, many of which use Alexa data for rating sites. Almost without fail the sales people first present the Alexa data as being solid and, when questioned, revert to (quite possibly genuine) ignorance on the subject.
Then, secondly, there is the slightly savier approach. Perhaps someone has memories of reading about a blogger bemoaning the difference between their server statistics and Alexa’s figures. Perhaps too they’ve looked closely at Google’s data and spotted how often the geographic popularity of sites matches up with the physical location of ISPs’ operations. Is that site really popular with the public in that area? Or is it a case of being popular with people all round the country, who happen to share an ISP which Google locates there?
But then you revert to the argument, “Yeah, it may not be perfect. But Google and Amazon are pretty trusted firms – and they’re much more successful than you or I. So I’ll carry on using their data but stick a caveat in now and again.”
That is a view I have some sympathy with. In the absence of resources to check otherwise, guessing that what Google or Amazon do is right is not a bad guess to make.
That then leads to the third level of argument. Looking at some hard data. So here is some.
I’ve taken Guido Fawkes’s blog during 2008 as my example because during that year, most months, he published the Google Absolute Unique visitors figures whilst also the blog has high enough traffic levels to register consistently with Alexa and Google Trends.
Here is how the data from the three compare (click here or on graphic for larger version):
The three data sources to track slightly different data, but the difference in the overall picture shown by Google and Alexa on the one hand and Google in another guise on the other should be troubling for anyone who wishes to rely on Alexa or Google Trends data.
Both Google Trends and Alexa present a picture of flat traffic levels through the year. Google Analytics presents a picture of strong growth. In all three cases the Y-axis starts at zero. Even with the end of year tail-off, Google Analytics has Guido Fawkes’s blog bringing in around 40% more traffic by the end of the year, whilst the picture from the other two graphs is of the blog ending the year much where it started (once the New Year holidays were over).
UPDATE: This blog post has a good example of the dangers of small data samples behind some Alexa information.