A conversation with ChatGPT, given a voice by Siri, shows AI’s promise and limitations

Mike Blinder, publisher of the trade journal Editor & Publisher and the host of its vodcast, “E&P Reports,” tried something unusual recently. He hooked up ChatGPT to Siri and conducted an interview about issues related to artificial intelligence and journalism. The result is like a smarter version of Eliza, a 1960s-vintage AI program that could carry on what seemed like a realistic conversation.

Blinder has become something of an AI evangelist, using to automate some of E&P’s editorial processes and asking ChatGPT to write bios of guests on “E&P Reports.” But AI has a long way to go in terms of carrying on an intelligent conversation that’s also spontaneous.

For instance, as you’ll hear, Blinder’s approach in interviewing ChatGPT is to lead the witness with long, information-packed questions that the chatbot can then use to scour the internet and come back with a plausible-sounding answer.

I also detected at least one error. In response to a question about the possibility that AI-powered search engines will harm news organizations by removing any incentive to click through, ChatGPT says:

When AI tools like Perplexity or Chat GPT provide detailed summaries of content without directing traffic back to the original source, it creates a real challenge for news publishers who rely on page views for ad revenue and subscriptions. I completely understand the concern.

The problem is that Perplexity actually does cite its sources, which differentiates it from ChatGPT and other competitors. It’s why I suggest to my students that Perplexity is a useful tool as long as they click through, and it’s why I use it as well.

Nevertheless, Blinder’s close encounter of the robotic kind is fun and interesting. You can watch it on YouTube or subscribe on any podcast app. Blinder’s been a guest on our “What Works” podcast about local news twice, and Ellen Clegg and I were on “E&P Reports” earlier this year to talk about our book, “What Works in Community News.”

Publishers sound alarm over Google’s AI search plans. But let’s dial it back a bit.

What does AI say about RFK Jr.’s anti-vax activism? Photo (cc) 2023 by Gage Skidmore.

The latest freakout inspired by artificial intelligence has hit the news business. On Tuesday, Google announced that it will merge its AI tool, Gemini, with its search engine. What that means for publishers is that someone seeking information about, say, Robert F. Kennedy Jr.’s anti-vax crusade will receive an answer right in Google rather than being prompted to click to a news source. Let’s try it:

First, I entered <Robert F. Kennedy Jr. anti-vaccine activism> in Google Search. The top five results were from The Associated Press (two), Scientific American, the Annenberg Public Policy Center and the PBS “NewsHour.” In other words, good, respectable sources of independent journalism.

Next, I entered the same search at Gemini and got this: “I’m still learning how to answer this question. In the meantime, try Google Search.” OK! But then I tried the same thing at the free version of ChatGPT, which provided the following answer:

Robert F. Kennedy Jr., son of former U.S. Senator and Attorney General Robert F. Kennedy and nephew of President John F. Kennedy, has been known for his anti-vaccine activism. He has been a prominent figure in the anti-vaccine movement, particularly regarding his views on the safety of vaccines and their alleged connection to autism.

Kennedy founded the Children’s Health Defense organization, which has been criticized for spreading misinformation about vaccines and promoting unfounded claims about their dangers. He has spoken at numerous events and rallies, advocating against vaccination and promoting alternative views on the subject.

It’s important to note that the overwhelming scientific consensus supports the safety and effectiveness of vaccines, and the alleged link between vaccines and autism has been thoroughly debunked by multiple scientific studies. Kennedy’s activism has been met with criticism from public health officials and medical professionals, who argue that his views are not supported by scientific evidence and pose a danger to public health.

I think we can agree that’s pretty good — and it includes exactly zero links to any of its sources, depriving some excellent news organizations of web traffic. I should note that, according to CNN media reporter Oliver Darcy, Google is promising to include links in Gemini, so it may end up looking something like Perplexity, an AI-powered search engine that provides plain-English answers and reliable links.

When I tried our RFK Jr. search at Perplexity, I got a very good answer — as good as ChatGPT, if not better — along with links to Wikipedia, The AP, the “NewsHour” and NBC News. But would I click on them? At the moment, I would, because of AI’s well-known proclivity for providing fake answers. At some point, though, that problem will be solved.

“This will be catastrophic to our traffic, as marketed by Google to further satisfy user queries, leaving even less incentive to click through so that we can monetize our content,” Danielle Coffey, chief executive of the News/Media Alliance, told Darcy. The alliance represents some 2,000 news publishers.

I also took a look at the internal metrics of the only news site I have access to: this one. According to Google Analytics, over the past month Media Nation received 40% of its traffic from “organic search” — that’s traffic from search engines, nearly all Google, that I didn’t boost by paying for ads on Google. And yes, that’s a lot. Next up was direct traffic (25.6%), organic social (21.2%) and referrals (12.1%), which are links from other websites.

Now, I happen to think that some of the lamentations we’re hearing from publishers are overblown. It’s fine to complain that Google is taking steps that will result in fewer clicks on your website. But how much money does that really bring in? These days, you’re likely to hit a paywall when you try to click through from a search. Programmatic ads on news sites are terrible and bring in very little money.

In the end, there is no substitute for building a relationship with your audience. For-profit publishers need to persuade their readers to become digital subscribers and local businesses to advertise. Nonprofits must convince their audience to become voluntary supporters and to raise money from underwriters, foundations, events and whatever else they can think of.

To use Media Nation as an example again: I currently have more than 2,300 subscribers who receive new posts by email. I consider those to be my most engaged readers. I don’t do much to monetize this site, although I have a modest paid supporter program, which, needless to say, you are invited to join. The future of news, though, is being built right now by serving our communities — not through Google search.

Leave a comment | Read comments