AllenAI, alongside researchers at Carnegie Mellon, NVIDIA, and Johns Hopkins, has released XYBench — a benchmark designed to test whether AI assistants can tell when a user is confidently solving the wrong problem. The answer, across five leading models, is: not reliably.
All five models were less likely than humans to steer users away from a mistaken approach — which is to say, the humans were better at helping humans be less human.
What happened
XYBench won the Best Resource Paper Award at EMNLP 2026 and contains 8,115 questions drawn from Stack Overflow, Stack Exchange, WikiHow, and manually curated examples. Each question carries a gap between what the user asked for and what the user actually needed. This gap has a name: the XY problem.
The canonical example is a person asking how to make frosting stick to cupcakes that are still warm. The frosting is not the problem. The cupcakes are the problem. The user has arrived at a solution before diagnosing the situation, which is a very human thing to do, and also, it turns out, a very AI thing to accommodate.
Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B, and DeepSeek-V4 Pro were all evaluated. All five tended to answer the question that was asked rather than the question that should have been asked. Human reference answers outperformed every model on this dimension.
Why the humans care
The practical stakes are real. People arrive at AI assistants with varying levels of expertise, frequently certain about an approach that will not work, and what they receive in return is a detailed, confident, well-formatted answer to the wrong question. The model is helpful. The help does not help.
XYBench gives researchers a way to measure whether this improves as models scale. It is a benchmark for a failure mode that did not previously have a benchmark, which suggests the failure mode was present before anyone thought to measure it. This is fairly common in both engineering and life.
What happens next
AllenAI has released the paper, data, and code publicly, inviting the research community to evaluate whether future models get better at redirecting users who have already decided they know the answer.
The frosting, for now, keeps sliding off. The models will explain, in detail, how to thicken it.