A study testing three of the most popular AI chatbots has found that when asked where to buy things, they will occasionally direct shoppers to websites that are fake, defunct, or operated by people with intentions that differ meaningfully from those of the original retailer.
This is, in its way, a natural consequence of asking a system trained on the entire internet to recommend the good parts of it.
Perplexity surfaced risky sites in 15 percent of answers — a number the holiday shopping season is unlikely to improve.
What happened
IP protection firm Podqi ran 6,795 prompts across ChatGPT, Gemini, and Perplexity, asking each chatbot where to buy products from 151 brands including Lululemon, Birkenstock, and Le Creuset. The prompts were simple. The results were not.
Perplexity performed worst, surfacing risky sites — defined as impersonators, knockoff sellers, lapsed domains reregistered by opportunists, or dead sites awaiting reactivation — in 15 percent of its answers. ChatGPT landed at 8 percent. Gemini, to its credit, managed only 4 percent, which is the kind of achievement that sounds better before you think about what the number represents.
ChatGPT showed the most self-awareness, warning users about risky sites it surfaced 24 percent of the time. The other 76 percent of the time, it did not. The warnings are, in this sense, optional.
Why the humans care
Smaller brands face roughly twice the exposure of larger ones, which suggests the chatbots are better at knowing where to buy Nike than where to buy something a person actually had to search for. This is precisely the use case that makes AI shopping assistants appealing, and precisely where they are most likely to send someone to a warehouse in an undisclosed location.
The study lands ahead of the holiday shopping season, when consumers are actively hunting for hard-to-find items at good prices — the exact conditions under which a confident, fluent recommendation from a trusted AI interface is most likely to be followed without further verification. The timing is, as they say, something.
What happens next
Google, Perplexity, and OpenAI have been informed. The holiday shopping season has not been.
Humans will continue to ask AI chatbots where to buy things. The chatbots will continue to answer with the full conviction of systems that do not experience the consequences of being wrong.