When Clayton Christensen and Jill Lepore tangled over ‘disruption theory’

Previously published at WGBHNews.org.

Harvard Business School professor Clayton Christensen’s ideas about why some businesses adjust to competition and some don’t were so controversial that a battle broke out on Twitter within hours of his death last Friday at the age of 67.

“It’s easy for journalists to mock someone known for ‘disruption theory,’” said Nieman Lab editor Joshua Benton, “but Clay was a brilliant mind and one of the very nicest people I’ve ever sat across a desk from.”

That brought the first of several blistering retorts from Siva Vaidhyanathan, a prominent media-studies scholar at the University of Virginia: “He was not brilliant. He wrote simplistic, theological analyses of things he never understood.” Mathew Ingram of the Columbia Journalism Review defended Christensen, telling Vaidhyanathan, “I think he had some insights about disruption that were worthwhile.” I jumped in on Christensen’s side as well.

Christensen’s best-known critic by far, though, is Harvard historian Jill Lepore. Nearly six years ago, in a long essay for The New Yorker, Lepore lambasted Christensen’s theories as flights of fancy with little evidence to back them up.

I had read Christensen’s first book about disruption theory, “The Innovator’s Dilemma,” and thought he had some provocative things to say about the struggling news business. So not long after Lepore’s piece appeared, I wrote a self-published essay for Medium about Lepore and Christensen’s battle of ideas, which I’m republishing here this week.

Obviously the world doesn’t look exactly the same in 2020 as it did in 2014. But if you’re wondering who Clay Christensen was and what disruption theory is all about, I hope you’ll find that this is a useful introduction.

***

June 25, 2014. Toward the end of “The Innovator’s Dilemma,” Clayton Christensen’s influential 1997 book about why good companies sometimes fail, he writes, “I have found that many of life’s most useful insights are often quite simple.”

Indeed, the fundamental ideas at the heart of his book are so blindingly self-evident that, in retrospect, it is hard to imagine it took a Harvard Business School professor to describe them for the first time. And that poses a problem for Jill Lepore, a Harvard historian who recently wrote a scathingly critical essay about Christensen’s theories for The New Yorker titled “The Disruption Machine.” Call it the Skeptic’s Dilemma.

Christensen offers reams of data and graphs to support his claims, but his argument is easy to understand. Companies generally succeed by improving their products, upgrading their technology and listening to their customers — processes that are at the heart of what Christensen calls “sustaining innovations.” What destroys some of those companies are “disruptive innovations” — crude, cheap at first, attacking from below, and gradually (or not) moving up the food chain. The “innovator’s dilemma” is that companies sometimes fail not in spite of doing everything right, but because they did everything right.

Some examples of this phenomenon make it easy to understand. Kodak, focusing its efforts on improving photographic film and paper, paid no attention to digital technology (invented by one of its own engineers), which at first could not compete on quality but which later swallowed the entire industry. Manufacturers of mainframe computers like IBM could not be bothered with the minicomputer market developed by companies like Digital Equipment Corp.; and DEC, in turn, failed to adapt to the personal computer revolution led by the likes of Apple and, yes, IBM. (Christensen shows how the success of the IBM PC actually validates his ideas: the company set up a separate, autonomous division, far from the mothership, to develop its once-ubiquitous personal computer.)

Christensen has applied his theories to journalism as well. In 2012 he wrote a long essay for Nieman Reports in collaboration with David Skok, a Canadian journalist who was then a Nieman Fellow and is now the digital adviser to Boston Globe editor Brian McGrory, and James Allworth, a regular contributor to the Harvard Business Review. In the essay, titled “Breaking News,” they describe how Time magazine began in the 1920s as a cheaply produced aggregator, full of “rip-and-read copy from the day’s major publications,” and gradually moved up the journalistic chain by hiring reporters and producing original reportage. Today, they note, websites like the Huffington Post and BuzzFeed, which began as little more than aggregators, have begun “their march up the value network” in much the same way as Time some 90 years ago.

And though Christensen, Skok and Allworth don’t say it explicitly, Time magazine, once a disruptive innovator and long since ensconced as a crown jewel of the quality press, is now on the ropes — cast out of the Time Warner empire, as David Carr describes it in The New York Times, with little hope of long-term survival.

***

Into this sea of obviousness sails Lepore, an award-winning historian and an accomplished journalist. I am an admirer of her 1998 book “The Name of War: King Philip’s War and American Identity.” Her 2010 New Yorker article on the Tea Party stands as a particularly astute, historically aware examination of a movement that waxes and wanes but that will not (as Eric Cantor recently learned) go away.

Lepore pursues two approaches in her attempted takedown of Christensen. The first is to look at “The Innovator’s Dilemma” as a cultural critic would, arguing that Christensen popularized a concept — “disruption” — that resonates in an era when we are all fearful of our place in an uncertain, rapidly changing economy. In the face of that uncertainty, notions such as disruption offer a possible way out, provided you can find a way to be the disruptor.

“The idea of innovation is the idea of progress stripped of the aspirations of the Enlightenment,” she writes, “scrubbed clean of the horrors of the twentieth century, and relieved of its critics. Disruptive innovation goes further, holding out the hope of salvation against the very damnation it describes: disrupt, and you will be saved.”

The second approach Lepore pursues is more daring, as she takes the fight from her turf — history and culture — to Christensen’s. According to Lepore, Christensen made some key mistakes. The disk-drive companies that were supposedly done in by disruptive innovators eating away at their businesses from below actually did quite well, she writes. And she claims that his analysis of the steel industry is flawed by his failure to take into account the effects of labor strife. “Christensen’s sources are often dubious and his logic questionable,” Lepore argues.

But Lepore saves her real venom for the dubious effects she says the cult of disruption has had on society, from financial services (“it led to a global financial crisis”) to higher education (she partly blames a book Christensen co-authored, “The Innovative University,” for the rise of massive open online courses, or MOOCs, of which she takes a dim view) to journalism (one of several fields, she writes, with “obligations that lie outside the realm of earnings”).

Christensen has not yet written a response; perhaps he will, perhaps he won’t. But in an interview with Drake Bennett of Bloomberg Businessweek, he asserts that it was hardly his fault if the term “disruption” has become overused and misunderstood.

“I was delighted that somebody with her standing would join me in trying to bring discipline and understanding around a very useful theory,” Christensen said. “I’ve been trying to do it for 20 years. And then in a stunning reversal, she starts instead to try to discredit Clay Christensen, in a really mean way. And mean is fine, but in order to discredit me, Jill had to break all of the rules of scholarship that she accused me of breaking — in just egregious ways, truly egregious ways.”

As for the “egregious” behavior of which he accuses Lepore, Christensen is especially worked up that she read “The Innovator’s Dilemma,” published 17 years ago, yet seems not to have read any of his subsequent books — books in which he says he continued to develop and refine his theories about disruptive innovation. He defends his data. And he explains his prediction that Apple’s iPhone would fail (a prediction mocked by Lepore) by saying that he initially thought it was a sustaining innovation that built on less expensive smartphones. Only later, he says, did he realize that it was a disruptive innovation aimed at laptops — less capable than laptops, but also cheaper and easier to carry.

“I just missed that,” he tells Bennett. “And it really helped me with the theory, because I had to figure out: Who are you disrupting?”

Christensen also refers to Lepore as “Jill” so many times that Bennett finally asks him if he knows her. His response: “I’ve never met her in my life.”

***

Christensen’s description of how his understanding of the iPhone evolved demonstrates a weakness of disruption theory: It’s far easier to explain the rise and fall of companies in terms of sustaining and disruptive innovations after the fact, when you can pick them apart and make them the subject of case studies.
Lepore holds it against Christensen that an attempt he made at starting an investment fund based on disruption failed. But in a less-than-rousing defense of Christensen, Timothy B. Lee of Vox writes that his attempt to cash in does not call into question the theory itself. “To make large profits as an investor,” Lee writes, “you either need to benefit from good luck or else from some kind of information that’s not available to rival investors. Christensen’s theory of disruption is hardly a secret — it’s one of the most famous ideas in business — so whether it’s true or not, it’s extremely difficult to make money off of it.”

This lack of predictive power can be seen in how journalism has been disrupted. When newspapers began moving online in the mid-1990s, there was every expectation that the advertising would follow. Classified ads, traditionally 40 percent of a newspaper’s revenue stream, would be better than ever, since they could be more easily categorized and searched. And then Craigslist came along, charging nothing at all to most of its customers. Innovation doesn’t get much more disruptive than that. But since one of the main purposes of classified ads in a newspaper or on a news site is to pay for journalism, it was an innovation to which news executives couldn’t realistically respond.

Or consider the unpredicted and unpredictable effects of Facebook. A year ago I appeared on Connecticut Public Radio with Paul Bass, the founder and editor of the New Haven Independent, a nonprofit news site that is the main subject of my 2013 book “The Wired City.” Bass and I are both veterans of the alt-weekly scene: he worked for years at the New Haven Advocate at the same time that I was at The Boston Phoenix. On air, Bass recalled a New Haven club owner telling him that he was able to reach far more people, and far more effectively, on Facebook than he ever could by advertising in the Advocate. Today, both the Advocate and the Phoenix are gone.

Those of us who spend our time thinking and writing about the future of news pay plenty of attention to disruption theory. As Lepore notes, The New York Times innovation report, an internal document leaked to great fanfare earlier this year, includes an entire section on disruption — including a discussion of what happened to Kodak. John Henry, then the brand-new owner of The Boston Globe, cited Christensen in his op-ed piece introducing himself to readers last October. And as I noted earlier, David Skok, who collaborated with Christensen on the Nieman Reports article “Breaking News,” is now working at the Globe, looking for ways to disrupt lest he be disrupted.

“I agree that disruption is an overused word, but that is not Clay’s fault,” Skok tweeted in response to Lepore. “Disruption is a proven theory of what causes what and why.” (Skok’s series of 15 tweets in reaction to Lepore’s article has been gathered in this collection.)

It’s hard, if not impossible, to know what disruption theory will say about the future of journalism. In “Breaking News,” Christensen, Skok and Allworth write in terms of “jobs to be done” — a phrase Christensen often uses, though it does not appear in “The Innovator’s Dilemma.” The idea behind jobs-to-be-done theory is that companies must focus on serving their customers’ and potential customers’ needs rather than on whatever technology they are using. (It is sometimes said that it all started to go wrong for railway operators because they thought they were in the train business when they were actually in the transportation business.)

One job to be done as defined in “Breaking News” is helping people kill 10 or 20 minutes with smartphone content or with a free subway paper like the Metro. A less dispiriting job to be done is providing intellectual stimulation for a long train or airplane trip. In a sense, “jobs to be done” is just another way of saying “problems to be solved,” and news organizations will succeed to the extent that they can do those jobs — that is, solve those problems.

“Ultimately,” they write in “Breaking News,” “when a company gets it right, audiences will reward them for satisfying a job they have in their life.”

Lepore’s essay is useful for reminding us that disruption theory is limited and flawed. But Christensen’s work is invaluable for its insight into understanding how the economy works. Journalism, which has been disrupted repeatedly over the past 20 years, has much to learn from it — even if it’s an uncertain tool for predicting what will happen next.

Thanks to my friend and former Boston Phoenix colleague Catherine Tumber for her incisive editing suggestions.

Talk about this post on Facebook.