ChatGPT can recommend and buy products, but it still needs humans

OpenAI recently launched Instant Checkout in ChatGPT, giving the AI chatbot the ability to both recommend and purchase products for users.

In the world Sam Altman and OpenAI imagine, ChatGPT acts as your personal shopper, travel planner, secretary, life coach, and tutor. And Instant Checkout brings us one step closer to that vision. But whether you realize it or not, those shopping recommendations are being cribbed straight from shopping journalists, product reviewers, and vloggers. Take away those human voices, and ChatGPT will be left with nothing but its hallucinations.

In the world of agentic AI and large-language models, human voices are becoming more important, not less. Yet those human voice are in danger of being drowned out entirely.

As one of those human voices (I'm a long-time journalist and product reviewer), I obviously stand to lose if people turn to ChatGPT over traditional media. But I don't think ChatGPT users realize just how much the chatbot depends on journalists, authors, and human creators of all kinds.

Instant Checkout is a coup for OpenAI

The Instant Checkout feature is a smart move by OpenAI. The inability of agentic AI tools to actually add items to your cart and purchase them has been a major limitation. Rumor has it, Amazon isn't thrilled about the prospect of rival AI bots replacing actual human shoppers. It would certainly threaten Amazon's lucrative advertising business, as brands pay billions each year to appear at the top of Amazon search results. Take away the human visitors, and you take away those ad dollars.

Yet by working directly with Shopify merchants for Instant Checkout, OpenAI has done an end-run around Amazon and other retailers. It's quite the coup for the company, and soon ChatGPT will be able to make purchases from 1 million merchants, without users ever leaving ChatGPT.

Since ChatGPT can't test a pair of headphones, set up a robot vacuum, or compare the taste of two espresso machines, how does it decide which product to recommend?

The same way ChatGPT knows anything — by scraping vast amounts of data from the internet.

The process is invisible to the end user, but LLMs like ChatGPT are trained on countless product reviews, YouTube video transcripts, and human-written content — including content from Mashable. To win the AI arms race, AI companies are ingesting all the content they can find, and they're doing this precisely because human-created content is so valuable.

(Disclosure: Ziff Davis, Mashable’s parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)

This copyrighted content is taken without permission or payment. In any other context, we call that stealing. And you don't steal something unless it has value.

AI companies are chipping away at the foundation of the internet economy

There's a catch-22 built into the AI industry. Chatbots like Gemini and ChatGPT are taking away the clicks, views, revenue, and advertising that publishers, journalists, YouTube creators, and other experts depend on. By laying claim to the sum of human knowledge and scraping the internet in the process, the AI industry is chipping away at the very foundation of the internet economy. If ChatGPT and Google can steal and summarize every single article published on sites like Mashable, then users never need to leave ChatGPT and visit our site. Already, many indie publishers and blogs are dying as the web changes. The Columbia Journalism Review calls it the Traffic Apocalypse.

But here's the catch: If the AI industry removes the incentive for real humans and journalists to publish new content online, then eventually the chatbots will run out of content for training. You may enjoy turning to ChatGPT for product recommendations or recipes, but those recommendations only have value if ChatGPT can keep absorbing the latest recipe blogs, cookbooks, and YouTube cooking videos.

So, how will AI chatbots provide answers in a post-human information economy? We have some idea.

Elon Musk has made it very clear that he believes X and Grok users can replace traditional journalism altogether. He's working on a Grokpedia right now. Instead of getting news from human journalists, with all their imperfections, Grok will simply scrape X to see what's happening in the world.

Meanwhile, Google is investing heavily in a partnership with Reddit. Reddit's visibility in Google Search has increased dramatically, and clearly, Google thinks Reddit will be a valuable source of training data for Gemini and AI Overviews.

Let's be clear: A world in which X and Reddit users are the primary source of information is a world on a direct collision course with the Idiocracy timeline — and I say that as a user of both platforms.

The irreplaceable power of human connection

No matter what the techno-utopians in Silicon Valley might think, there's still no replacement for actual human experience.

Already, we're seeing the limits of chatbots when they can't simply scrape the answer from human sources. During the Charlie Kirk shooting, AI chatbots struggled to provide answers to the most basic questions — like, "Is Charlie Kirk alive?" — and it's easy to see why. The event was so new that LLMs couldn't accurately predict the answers to users' questions.

We'll see the same problem when a new product launches. After the iPhone 17 Pro launch, reviewers and customers quickly discovered that the new aluminum body of the phone was extremely prone to scratching — exactly the kind of first-hand experience you need when making a purchasing decision.

Vivek Shah, the CEO of Mashable publisher Ziff Davis, recently tackled this topic in an appearance on the Channel media podcast. He summarized how many journalists feel about the AI era, when every word you write can (and almost certainly will be) taken by a chatbot.

"I think in the end, as humans, we read, we listen, we watch, at least in the context of content and media," Shah said. "And I still think we prefer words and sounds and videos from humans. Do I think that the robots will eat into some of that? I do. I think the real question is when do we crave a human voice?"

Shah added, "But I think that if the robot distillation takes away voice and tone and narrative and sources and all the things that I think make human-created content great, I think you’re going to want to go and dig in deeper."

And what about topics that demand in-depth investigation? AI chatbots can't interview sources. They can't contact experts. They can't call bullshit. They can only scrape, predict, and summarize, with a healthy side serving of ego stroking. That's a valuable skillset, no doubt, but it's also a very limited one.

The goal of most apps is simple — to keep your attention on the app for as long as possible (hence, the ego stroking).

And ChatGPT is very good at holding your attention. But as AI chatbots take on a larger role in society, don't forget to step outside the ecosystem.

It's worth remembering that ChatGPT depends on journalists and reviewers to answer your questions — and sites like Mashable depend on you.

This article reflects the opinion of the writer.