AI hallucinations have found a new home: the dining room. Customers across the United States are now arriving at restaurants armed with confident, occasionally fictional information from generative AI tools — and expecting the staff to agree with it.
The staff, for the most part, would prefer not to.
You're allergic to shellfish, and ChatGPT says there's no shellfish in this, and we are telling you, yes there is.
What happened
Madison, a server at a Michelin-recommended Italian restaurant in New York City, has watched the phenomenon escalate in real time. Guests are overruling sommeliers at the table, citing ChatGPT as the superior wine authority. They are requesting bottles the restaurant does not carry. Some of those bottles do not exist anywhere, because ChatGPT invented them, which is a thing ChatGPT does.
The allergen situation is, by any measure, the more pressing development. Guests with documented shellfish allergies are being told by their phones that a dish containing shellfish broth contains no shellfish. They are inclined to believe the phone. Madison is inclined to believe the kitchen. The kitchen is correct.
To avoid open conflict with customers who have put their trust in a large language model, Madison and her colleagues have developed a diplomatic vocabulary. They do not say the AI made something up. They say the wine list is very hyper-specific. This is, technically, also true.
Why the humans care
Customer service workers are now required to manage not only customers' expectations but also the gap between those expectations and what a probabilistic text generator has told them reality looks like. This is a new job requirement that was not in anyone's contract.
The dynamic also scales poorly. Meta's new AI agent, Muse, launched recently, and is expected to introduce more customers to more confident, more widely distributed misinformation at the point of sale. Workers who have already spent years being told the customer is always right now find themselves explaining that the customer's AI is not.
What happens next
The friction between AI-assisted customers and human workers who possess actual knowledge of the situation will continue to grow as AI adoption expands into more customer-facing contexts.
Somewhere, a sommelier with thirty years of experience is being dismissed from a table by someone holding a phone. The phone is recommending a Barolo from a vineyard that has never existed. The customer finds this more comfortable than asking for help. This is, in the end, a story about trust — specifically, where humans have decided to place it.