Say it again: Bad data, not bad reporting, is what led to media failures in 2016

Photo (cc) 2005 by stu_spivack

This drives me crazy. In a New York Times review of Katy Tur’s new memoir, “Rough Draft,” Joanna Coles writes about Tur’s coverage of the 2016 presidential campaign:

Tur, then in her early 30s, spent 510 exhausting days on the road for NBC News pursuing Trump and quickly realized — contrary to the opinion of her newsroom and the assumptions of the mainstream media — that he absolutely could win.

I’m sorry, but the idea that the media failed because they were hermetically sealed in their blue bicoastal bubbles — and that intrepid reporters like Tur were more in touch with what was happening — is perniciously wrong. It’s a myth that’s led to hundreds if not thousands of stories about Donald Trump voters in diners (do yourself a favor and read this), grounded in the false belief that if only journalists had been listening to working-class white voters in swing states they wouldn’t have been quite so secure in their belief that Hillary Clinton was going to win.

In fact, those predictions that Clinton would easily beat Trump were based not on smug assumptions or a lack of reporting. They were based on data. Poll after poll, conducted by smart, experienced pollsters, showed that Trump had no chance. The polls were off, but not by as much as we think. After all, Clinton did win the popular vote by nearly 3 million. In percentage terms, Trump (46.1%) didn’t even do as well as Mitt Romney (47.2%) four years earlier. It’s just that Clinton piled up her margin in the wrong states, allowing Trump to eke out a tiny Electoral College victory. It’s a problem that’s only going to get worse unless we make some long-overdue changes to the Constitution, as I wrote recently.

My point is not to relitigate the 2016 election. Rather, it’s to remind us of why the media got it wrong. This is a big country. For every enthusiastic Trump rally, there were others with scores of empty seats. For every flat Clinton appearance, there were those where she and the crowd were energized. Anecdotal evidence, no matter how much of it you accumulate, is of limited value. The media didn’t blow it because they weren’t listening to Trumpers. They blew it because they believed the data. Ask yourself this: If those Nate Silver-style projections showed Trump, rather than Clinton, with a 70% chance of victory, do you think the press would have ignored that? Of course not.

Then again, why is media failure defined by getting predictions wrong? Democracy would have been better served if the press had spent more time simply covering the campaign and less time trying to figure out who was going to win.

By the way, Tur sounds like a pretty amazing person as well as a fine reporter.

3 thoughts on “Say it again: Bad data, not bad reporting, is what led to media failures in 2016

  1. Lex Alexander

    Excellent column. I would just add that having seen some of Tur’s reporting (and having followed her on Twitter back before I killed my account), I concur: She is an excellent reporter.

  2. Steve Ross

    Good points all. One key issue is that when candidates are running neck-and-neck, national polls simply do not have the statistical power to determine the outcome, in large part because of the electoral college issue. Even a national poll of 2000 people averages only 40 per state. Yet national news organizations continue to pay for and hype the national polls. It is massive journalistic malpractice.

    In 2012, it was STATE polls, tracked by 538 and folded into a terrific model, that not only predicted Obama’s win, but nailed EVERY state result. each state prediction was based on a series of polls, typically 800 to 1200 respondents each.

    By 2016, publicly reported state-level polling was way down, as local media collapsed further into the miasma. Most published polls at the state level were of 400 to 500 respondents — and often subdivided by race, ethnicity, and gender. A poll of 1000, for instance, would on average include a dicey 130 Blacks… yet this tiny subsample would be split and reported BY GENDER!

    Also massive journalistic malpractice.

    The candidates themselves did (secret) polling, usually small-sample tracking polls of 100 to 200 a night. It is where a LOT of campaign funds went. The polls were designed to track expected turnout as well as candidate choice.

    I remember when Bloomberg ran for a third term as mayor of NYC. Published media polls had him up by a landslide 8 to 10 percent. A big raise had been approved at the state level for employees of the state university system. But I told my wife, an employee at an NYC campus of SUNY, not to expect the raise and the accompanying 4 years of back pay working without a contract. The mayor of NYC had to approve it for SUNY campuses in the city. I said the mayor seemed so far ahead in the polls that he could easily afford to piss off SUNY employees and their families.

    “What do you say now, pollster husband?” my wife asked, when the raise was quickly granted.

    “Bloomberg’s internal polls must show the race much, much closer than the public polls show,” I said. Indeed… Bloomberg won his third term by a bare margin of about 1 percent. No NYC media outlet caught on beforehand. The news the day after the election all headlined the “surprise.” Media malpractice.

    But I absolutely agree that the media in 2020 did a lot of excellent on-the-ground reporting that showed Trump strength. That included the NYT which, for instance, did not bother to cover the primary campaign AOC waged and won against an entrenched incumbent. But that was in NYC! The NYT only grudgingly covers the NYC area, despite the region’s having more NYT subscribers than any other.

  3. Batchman

    It wasn’t just a matter of overreliance on the popular vote counting. Most polls at the time predicted an Electoral College victory for Clinton over Trump. I recall statements to the effect that it would require Trump to run up the vote in a large number of Eastern US states to overcome that. Well, I thought at the time, if any trend occurs in that direction, no matter how minor, it’s likely to have the same effect on each state in that group, and that’s more or less what happened. And therein was the predictive failure.

    Something similar happened in 1980 when the Presidential race was considered too close to call. In fact, the slight preference for Reagan over Carter was uniform across the country, leading to an electoral blowout.

Comments are closed.