Regarding Alexa
(January 12,
2004)
- Last week, we ran Brian Kreuger's Job
Board rankings, as we often do. Predictably, we received the avalanche of
mail complaining about Alexa, Brian's biases, flaws in the list, missing job
boards, the general unfairness of life and a variety of other grumpy stuff.
Nothing stirs the competitive juices like a ranking list.
Alexa,
a free service provided by the folks at Amazon, tracks the viewing behavior of
people who have installed the Alexa toolbar. Although there are some modest
privacy concerns (Alexa knows your surfing behavior), we've always believed that
the Alexa toolbar provides a reasonable return. With Alexa installed in your
browser, you can immediately see a range of interesting things about the
websites you are visiting.
Alexa provides ranking information (how does the
site you are viewing compare with all other websites), contact information (from
the Domain Name Records), age of the website, a quality ranking (based on the
votes of Alexa surfers), related links (where else do people who visit the site
go?), inbound links (a distinct measure of site quality) and other useful data
about the site. It's a necessary part of understanding a particular website in a
larger context.
And, the price is right. Although you must allow
Alexa to monitor your online behavior, there is no additional charge for the
service. It's free.
Large web operations pay as much as $15,000 per
month for similar data.
Alexa is biased. Since its ranking information
is compiled from a small set of users (several million), there are known 'holes
in the data'. Alexa only looks effectively at a domain name, so numbers for
operations like the CareerJournal are somewhat skewed.
Interestingly, the large commercial services are
also biased. On the web (as is true with print, radio and television), audience
measurement is as much art as it is science. Infrastructure issues (cacheing, in
particular) always skew the measurements. Although it seems like a computer
based medium ought to be very measurable, we have never seen two measures (using
the same system) that produced the same results.
That's the thing about any measurement system.
In the absence of accuracy, you have to settle for consistency. It's true in
performance measurement in organizations and its true for job board rankings.
The other truth about Alexa is that it is easy
to manipulate. Readers pointed out a number of obvious examples of data
manipulation. There's a Job Ad Distribution company whose ranking is in the
3,000 range while all of the competition hovers closer to 100,000. Known slime
balls, the company manipulates Alexa data as a way of trying to fool its
customers. The scam is obvious to anyone who takes a look at the data.
At the same time, it's fair to say that our
advocacy for Alexa contains a quiet bias as well. When our readers install the
toolbar, our ranking goes up.
Again, any measurement system is subject to
aggressive and/or subtle manipulation. In our kitchen is a wall on which each
family member's height has been measured with a pencil line over the years. At
measurement time, everyone stands a little taller.
Our archives are full of articles about the
features, benefits and uses of Alexa. As flawed as it is, it's readily available
and without an alternative. We recommend that you install the toolbar.
John
Sumser