Among the more difficult challenges I faced when I was researching “The Wired City” was trying to figure out the influence — and thus the effectiveness — of the New Haven Independent, the nonprofit online-only news site that is the major focus of the book. So I was intrigued when NPR reported last week that the folks who run Charity Navigator hope to unveil effectiveness ratings for nonprofits in 2016. I suspect it won’t be easy.
Nonprofit news presents special challenges. You can measure a food pantry’s effectiveness by how many families get fed, or an orchestra’s reach in terms of attendance, educational programs and the like. But how do you measure the effectiveness of a news organization?
The way I tried to answer that question was to look at numbers, but to look beyond the numbers as well. Certainly the numbers couldn’t come close to telling the whole story. The Independent’s main competitor is the regional daily newspaper, the New Haven Register, whose website receives considerably more traffic than the Independent’s (two to three times as much, according to Compete.com‘s very rough numbers) — and which, unlike the Independent, has a daily print edition. How can the Independent possibly exercise the same amount of influence?
It can’t. But that doesn’t mean that it’s ineffective. Since it’s not a mass-circulation outlet, the Independent relies on reaching civically engaged people — city leaders, neighborhood activists and anyone who takes an interest in what’s going on in New Haven on a daily basis.
In a 2009 interview, Michael Morand, an associate vice president at Yale, described the Independent’s readers as “active voters, elected and appointed officials, opinion makers, civic activists as measured by people who are on boards, leaders of block watches and other neighborhood organizations.” Mayor John DeStefano told me in 2011 that he considered the Register to be “the dominant media,” but added that the Independent serves “those that follow city government and community-based activities.”
One issue stood out for me as an example of how the Independent could be effective while reaching a niche audience. In the fall of 2010 the Independent — following the lead of the Yale Daily News — began reporting on incidents in which police officers ordered people not to video-record them while they were making arrests and engaging in other activities. The Independent reported on the story for months, and revealed that one man on a public sidewalk had his cellphone confiscated and the video erased; the man was charged with interfering with police.
The Independent’s reporting led to real reform. An investigation was conducted. Mayor DeStefano and the police chief at that time, Frank Limon, ordered that officers stop harassing people trying to video-record their actions as long as they weren’t interfering in police business. Mandatory training in how to respond to video-camera-wielding members of the public was added to offerings at the police academy — training that Independent founder Paul Bass and I observed during a visit to the academy. And state Sen. Martin Looney, a New Haven Democrat, introduced legislation to make it clear that such recording was legal.
The Independent also builds influence through public events — community forums such as political debates and other events that are webcast and covered by other media as well. A particularly prominent example was a school forum in late 2010 starring education-reform critic Diane Ravitch.
I don’t think numbers tell the story, but here are a few:
- According to the Independent’s internal counts from Google Analytics and Mint, the site receives between 120,000 and 140,000 unique visitors a month.
- The Independent’s Facebook page has been “liked” 3,443 times as of this morning.
- The Independent’s Twitter feed, @NewHavenIndy, has 4,486 followers.
- The site’s weekly news email has about 4,000 subscribers, according to Bass — up about 1,000 over the past year.
- A separate arts email has about 5,000 subscribers.
And here’s a cautionary tale for publishers who rely on outside services like Facebook, which have their own agendas: Bass told me that a change in policy may be responsible for less traffic coming to the Independent from Facebook. “Now they don’t automatically show your posts to all your followers. They show them to a sample, hoping you’ll pay to reach more of your people,” Bass said in an email last week. “So we’ve seen a drop in Facebook traffic as a result. (Which maybe quite fair. We use them free as paperboys.)”
When I asked Bass how he thought funders and prospective funders could measure the effectiveness of nonprofit news, his response was that he wasn’t sure. So what attempts have his funders made? “They look at what we publish, how people interact with it, how they experience the site’s impact living and working in the area,” Bass said. “No one has asked us for numbers.”
According to the NPR story, reported by Elizabeth Blair, some nonprofits are resisting Charity Navigator’s attempt to measure how well they’re doing. And there’s no question it will be difficult — maybe impossible — to come up with a fair system that everyone will agree on.
But it seems worth trying, especially in journalism. With traditional for-profit news, the relationship between revenue and quality is only tangential. News organizations are judged by how well their advertising moves products and services. You’d like to think that advertisers would rather be associated with quality news than the alternative, but that’s not necessarily the case.
With nonprofit news, the better the journalism, the more influence it will have — and the more attractive it should be to funders.
Photo (cc) by Anne and published under a Creative Commons license. Some rights reserved.
Discover more from Media Nation
Subscribe to get the latest posts sent to your email.
Dan – here’s another question for you. How do you rate the effectiveness of the evaluators?
I run a non-profit human services agency for the blind. We are listed on the Giving Commons web site from the Boston Foundation, and on GuideStar. While I had seen emails and articles from Charity Navigator, I had never been to their web site so I clicked through. We are listed as an ‘unrated’ organization in good standing with the IRS. My problem with this is that I don’t ever remember being asked to BE rated by them. To me, they are the equivalent of Google or Yelp, web sites that solicit your information by publishing what is in the public domain, asking that you ‘claim your site’.
If you visit the Giving Commons, you will see that their questionnaires and forms represent at least a couple days of work merely to fill out, let alone keep updated. A new innovation on many is to describe your mission in 2,000 characters or less. Not words, CHARACTERS – and that includes spaces and punctuation. It has turned me into an ampersand addict. And there are dozens, if not hundreds, of these web sites, and some are just Consumer Reports/Better Business Bureau wannabes that hope to garner cuts from donations they collect and pass on.
As far as effectiveness goes – my clients are blind, and overwhelmingly elderly. About 10% are even computer literate, and even fewer are on-line (although that is changing with time). While I have dozens of cards and painfully written notes of thanks, none of my people are going onto a ranking web site. I don’t mind the informational and legal status presentations on such a site – that is merely prudence by a donor. But I am leery of an effectiveness ranking on par with Angie’s List, since it assumes that all people being helped by a charity are both computer literate and on-line.
@Cynthia: Excellent questions. Yesterday I read through a Knight Foundation report on the state of nonprofit news sites. I had several problems with the methodology, the most important of which was problems in making the right distinctions between foundation grants and high-net-worth individuals who, for tax purposes, use their foundations when making donations. It make the New Haven Independent appear to be more dependent on foundations than it really is.