Lucas Graves reports in the Columbia Journalism Review that the state of the art in counting online audiences remains abysmal.
Graves notes that statistics compiled by two of the leading services that rely on user surveys — Nielsen and comScore — can differ wildly. And, as every website operator knows, those numbers are often far lower than the numbers they get from Google Analytics and other internal measurements.
Why is it so hard? User reports are notoriously unreliable, and website operators have been complaining for years that the Nielsens are useless for measuring what people do when they’re at work. But the seemingly greater accuracy afforded by simply counting incoming traffic raises other problems: users who clear their cookies are counted every time they return; search engines that robotically visit sites are counted as users; and people who use more than one computer are counted multiple times.
My first encounter with the difficulties of counting came in 2007, when I was reporting this story for CommonWealth Magazine. I learned that the internal statistics at both the Boston Globe and the Boston Herald showed their Web audiences were three times larger than what Nielsen was reporting.
It hasn’t gotten much better since then. For instance, the New Haven Independent, a non-profit online news organization that I follow closely, was attracting some 70,000 unique visitors a month in 2009, according to founder and editor Paul Bass. That grew to 197,000 in September 2009, the month that Yale University graduate student Annie Le was murdered.
Yet according to Compete.com, the Independent was attracting just 25,000 to 30,000 uniques a month, a number that grew to 70,000 in September 2009. In other words, Bass’ internals placed the Independent’s traffic at about two and a half times what Compete.com was reporting, similar to what I had found with the Globe and the Herald two years earlier.
Then there’s the whole matter of “unique visitors per month” somehow becoming the most important measure of Web traffic. Wouldn’t you rather know how many people visit every day?
I’ve settled on Compete.com as being the easiest, most reliable free service available. It is supposedly based on surveying the behavior of some 2 million people. One thing I like is that its numbers seem reasonable. For instance, it regularly places the Globe’s Boston.com at roughly (very roughly) 5 million uniques per month, which is very close to the Nielsen figure.
Then again, maybe counting isn’t much better in other forms of media. As Graves’ CJR article points out, it’s easy to count how many newspapers are sold, but impossible to tell how many people read them. And television and radio audience measurements have been controversial for years.
So what is the solution? There may not be one, at least if “solution” is defined as something that is mathematically accurate. If people are reading and talking about you, you’ll know.
Discover more from Media Nation
Subscribe to get the latest posts sent to your email.
The significant point Graves’ piece makes is that it is in publishers’ interests for the various traffic measures NOT to agree. Thus, a publisher has more opportunities to shade their site traffic in the best possible light.
@Mike: True, but that’s not the point of my post, and I’m not sure I agree with him on that.
Maybe the solution is to stop counting?
@Dan: Ah, not the point of your post, but it was the point of his article.
The problem with just stopping counting, is what metrics do commercial sites use to determine their viability to potential advertisers, and what rates they can charge for ads?
I prefer quantcast over compete. It’s much closer to agreeing with GA, which I trust.
Compete is so wildly wrong they should be ashamed for continuing to publish stats.
As for stats and advertising — there are things much more important than anything other than ballpark stats to selling an ad that getting it just right doesn’t really matter.
@Howard: The problem with Quantcast is that it’s worthless if you’re dealing with a site that hasn’t registered with them. The figures for the Batavian look right on the mark, but the figures for the New Haven Independent (not registered) are just nuts.