The primitive art of measuring online audience

Lucas Graves reports in the Columbia Journalism Review that the state of the art in counting online audiences remains abysmal.

Graves notes that statistics compiled by two of the leading services that rely on user surveys — Nielsen and comScore — can differ wildly. And, as every website operator knows, those numbers are often far lower than the numbers they get from Google Analytics and other internal measurements.

Why is it so hard? User reports are notoriously unreliable, and website operators have been complaining for years that the Nielsens are useless for measuring what people do when they’re at work. But the seemingly greater accuracy afforded by simply counting incoming traffic raises other problems: users who clear their cookies are counted every time they return; search engines that robotically visit sites are counted as users; and people who use more than one computer are counted multiple times.

My first encounter with the difficulties of counting came in 2007, when I was reporting this story for CommonWealth Magazine. I learned that the internal statistics at both the Boston Globe and the Boston Herald showed their Web audiences were three times larger than what Nielsen was reporting.

It hasn’t gotten much better since then. For instance, the New Haven Independent, a non-profit online news organization that I follow closely, was attracting some 70,000 unique visitors a month in 2009, according to founder and editor Paul Bass. That grew to 197,000 in September 2009, the month that Yale University graduate student Annie Le was murdered.

Yet according to Compete.com, the Independent was attracting just 25,000 to 30,000 uniques a month, a number that grew to 70,000 in September 2009. In other words, Bass’ internals placed the Independent’s traffic at about two and a half times what Compete.com was reporting, similar to what I had found with the Globe and the Herald two years earlier.

Then there’s the whole matter of “unique visitors per month” somehow becoming the most important measure of Web traffic. Wouldn’t you rather know how many people visit every day?

I’ve settled on Compete.com as being the easiest, most reliable free service available. It is supposedly based on surveying the behavior of some 2 million people. One thing I like is that its numbers seem reasonable. For instance, it regularly places the Globe’s Boston.com at roughly (very roughly) 5 million uniques per month, which is very close to the Nielsen figure.

Then again, maybe counting isn’t much better in other forms of media. As Graves’ CJR article points out, it’s easy to count how many newspapers are sold, but impossible to tell how many people read them. And television and radio audience measurements have been controversial for years.

So what is the solution? There may not be one, at least if “solution” is defined as something that is mathematically accurate. If people are reading and talking about you, you’ll know.

Boston.com editor blames Celtics

Boston.com editor Dave Beard says that his site’s readership numbers are down over the previous May and June in large part because of the Boston Celtics’ early flameout. Since the Kevin Garnett-less Celts were eliminated on May 17, that explanation definitely makes some sense. Beard writes:

Two words: Boston Celtics.

No rolling rally. No euphoria. No heart-stopping NBA finals, sadly, with photo-friendly celebs packing the Garden. No May-to-June buildup for such a record season.

That said, despite a 40 percent dropoff from Sports in June pageviews, we made up most of it with News and Arts and Entertainment. By internal measurement, we came very close to our record June number for pageviews and for unique users, and we showed 11 percent more visits that the preceding June.

And, not to be a salesman on you, but July is looking very strong.

In short: We don’t read too much into a month or two of the widely variable and smallish samplings of Nielsen, as you mentioned in your lede. But we’re not relaxing one bit.

Boston.com’s missing readers

Web readership numbers tend to fluctuate so wildly that only a fool would try to read any deeper meaning into month-to-month changes.

Still, it’s hard not to notice that the number of unique monthly visitors to Boston.com, the Boston Globe’s Web site, has dropped considerably over the past two months, according to Nielsen Online figures provided by a reliable source who asked not to be identified.

After hitting an all-time peak of 8.5 million visitors in January of this year, perhaps tied to President Obama’s inauguration, the figure plunged all the way to 4 million in June, a drop of more than 23 percent over the previous June and the lowest number in two years.

Last June, then-senators Barack Obama and Hillary Clinton were engaged in the final stages of their hard-fought battle for the Democratic presidential nomination, and I’ve been told that Boston.com’s political coverage does well in search engines and aggregators. So that could be an explanation.

Or maybe it’s Red Sox fatigue — it’s possible that the Sox have been so good for so long that casual fans are checking in less frequently than they used to. Could it be the Globe’s labor unrest? (Unlikely, though it’s interesting that readership figures for the Globe’s corporate cousin NYTimes.com were also down by 21 percent in May.) Or maybe it’s just one of those things.

Based on Nielsen’s May figures, Boston.com is now the ninth-ranked newspaper Web site overall — down from sixth for all of 2008, though it’s still the most widely read regional newspaper site in the United States*.

Among all news sites, including perennial ratings leaders MSNBC.com, CNN.com and Yahoo News, Boston.com now ranks 32nd, down from 17th in January.

What follows are Boston.com’s unique monthly visitors over the past two years. Percentages are increases and decreases over the previous year.

  • June 2009: 4,020,000 (-23.2%)
  • May 2009: 4,397,000 (-11.4%)
  • April 2009: 5,888,000 (+33.0%)
  • March 2009: 5,742,000 (+37.2%)
  • Feb. 2009: 5,659,000 (+15.4%)
  • Jan. 2009: 8,535,000 (+64.3%)
  • Dec. 2008: 4,086,000 (-6.4%)
  • Nov 2008: 5,436,000 (+12.3%)
  • Oct 2008: 6,133,000 (+11.4%)
  • Sept 2008: 8,610,000 (+121.5%)
  • Aug 2008: 4,479,000 (+3.9%)
  • July 2008: 4,891,000 (+21.4%)
  • June 2008: 5,233,000 (+23.0%)
  • May 2008: 4,962,000 (+22.9%)
  • April 2008: 4,428,000 (+6.2%)
  • March 2008: 4,184,000 (-1.4%)
  • Feb. 2008: 4,904,000 (N/A)
  • Jan. 2008: 5,194,000 (N/A)
  • Dec. 2007: 4,364,000 (N/A)
  • Nov. 2007: 4,839,000 (N/A)
  • Oct. 2007: 5,506,000 (N/A)
  • Sept. 2007: 3,887,000 (N/A)
  • Aug. 2007: 4,311,000 (N/A)
  • July 2007: 4,029,000 (N/A)
  • June 2007: 4,254,000 (N/A)
  • May 2007: 4,038,000 (N/A)
  • April 2007: 4,171,000 (N/A)
  • March 2007: 4,245,000 (N/A)

*I’ve been asked how Boston.com can be considered the most-read regional newspaper site when the Los Angeles Times, the Chicago Tribune, the New York Post and New York’s Daily News are all ahead of it.

My answer is that the LA Times has long been considered a national paper. Indeed, Slate includes it as one of the five papers it summarizes in its “Today’s Papers” feature. (The others are the New York Times, the Washington Post, the Wall Street Journal and USA Today.)

It’s only recently that Boston.com’s readership dropped below those of the other three papers, so perhaps I’ll have to rethink my “most-read regional newspaper site” formulation. Both New York and Chicago are huge metropolitan areas that dwarf Greater Boston. Yet the Tribune, the Post and the Daily News are all more regional than they are national.

I’ll have to ponder that for a bit.

Nielsen Online numbers for April

Just got a look at the Nielsen Online numbers for April, passed along by a Media Nation reader. A few things leap out:

  • Of the top five news sites, none belongs to a newspaper. MSNBC.com is number one, with nearly 40.1 million unique visitors for the month, followed by Yahoo News (39.1 million), CNN.com (37.2 million), AOL News (23.4 million) and Fox News (18.1 million, but up nearly 67 percent over the previous year).
  • The New York Times, which fell to sixth (16.5 million), was down nearly 18 percent over March and nearly 8 percent over the previous year.
  • The Boston Globe’s site, Boston.com, ranked 23rd in overall news sites (5.9 million), up 33 percent over the previous year. Boston.com remains the most successful regional newspaper site in the country.
  • The Drudge Report, at 41st (3.2 million), has really seen better days, coming in well behind Web-only sites such as the Huffington Post (16th, 8.9 million), Slate, the Times of London and the Guardian (online-only in the U.S., at least).

Wish I had a link for you. And thanks to Mr. or Ms. X for sending this along.