Lucas Graves reports in the Columbia Journalism Review that the state of the art in counting online audiences remains abysmal.
Graves notes that statistics compiled by two of the leading services that rely on user surveys — Nielsen and comScore — can differ wildly. And, as every website operator knows, those numbers are often far lower than the numbers they get from Google Analytics and other internal measurements.
Why is it so hard? User reports are notoriously unreliable, and website operators have been complaining for years that the Nielsens are useless for measuring what people do when they’re at work. But the seemingly greater accuracy afforded by simply counting incoming traffic raises other problems: users who clear their cookies are counted every time they return; search engines that robotically visit sites are counted as users; and people who use more than one computer are counted multiple times.
My first encounter with the difficulties of counting came in 2007, when I was reporting this story for CommonWealth Magazine. I learned that the internal statistics at both the Boston Globe and the Boston Herald showed their Web audiences were three times larger than what Nielsen was reporting.
It hasn’t gotten much better since then. For instance, the New Haven Independent, a non-profit online news organization that I follow closely, was attracting some 70,000 unique visitors a month in 2009, according to founder and editor Paul Bass. That grew to 197,000 in September 2009, the month that Yale University graduate student Annie Le was murdered.
Yet according to Compete.com, the Independent was attracting just 25,000 to 30,000 uniques a month, a number that grew to 70,000 in September 2009. In other words, Bass’ internals placed the Independent’s traffic at about two and a half times what Compete.com was reporting, similar to what I had found with the Globe and the Herald two years earlier.
Then there’s the whole matter of “unique visitors per month” somehow becoming the most important measure of Web traffic. Wouldn’t you rather know how many people visit every day?
I’ve settled on Compete.com as being the easiest, most reliable free service available. It is supposedly based on surveying the behavior of some 2 million people. One thing I like is that its numbers seem reasonable. For instance, it regularly places the Globe’s Boston.com at roughly (very roughly) 5 million uniques per month, which is very close to the Nielsen figure.
Then again, maybe counting isn’t much better in other forms of media. As Graves’ CJR article points out, it’s easy to count how many newspapers are sold, but impossible to tell how many people read them. And television and radio audience measurements have been controversial for years.
So what is the solution? There may not be one, at least if “solution” is defined as something that is mathematically accurate. If people are reading and talking about you, you’ll know.