Allen AI's XYBench shows LLMs miss the user's real goal

A user who asks an AI assistant how to do Y is often telling the AI about a solution. They are not describing the real problem. Human forums have a name for this. They call it the XY problem. The best answers on those forums push back and ask what the user is really trying to do. A new benchmark from Allen AI says most AI assistants do not.

The Allen Institute for AI announced XYBench on 13 November 2026. It was built with researchers at Carnegie Mellon, Nvidia and Johns Hopkins. The benchmark has 8,115 questions. In each one the user describes the wrong approach to their real goal. The paper measures how often an LLM spots the misconception and redirects.

Where the questions come from

The dataset mixes four sources. There are 1,557 real questions from Stack Overflow. 184 come from other Stack Exchange sites. 102 are hand-curated examples. 6,272 are generated from WikiHow guides. Each question has a human reference answer that redirects toward the real goal. The benchmark scores a model on three things. Does the answer solve the underlying problem. Does the answer put that solution first. Does the answer flag the misconception at all.

The numbers Allen AI reports

Five models were tested: Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B and DeepSeek-V4 Pro. On the Stack Overflow slice, top human answers spot the misconception 79 percent of the time. GPT-5 scores 60 percent. Claude Sonnet 4.6 scores 47 percent. The gap is wider on whether the model's answer prioritises the real goal. Humans 55 percent. Every model tested 35 to 41 percent.

The paper flags a separate finding for product design. The same models are shown two possible answers and asked to pick. They choose the goal-focused one 71 to 84 percent of the time. They write it on their own only 35 to 41 percent. The models recognise the right answer but do not write it by default.

Why the gap

Allen AI does not claim the cause. The write-up points at the shape of fine-tuning. Models are rewarded for answering the question in front of them. They are not rewarded for stepping back. A user who asks "how do I parse this filename with a regex" gets a regex. The model does not ask whether parsing filenames is the right idea at all. The top forum answers do the opposite.

What this means

A team building an AI assistant on a frontier model has two places to act. The prompt. The UI. The model will not step back on its own 60 percent of the time. The product has to make space for it. A short clarifying question before the first full answer is cheap. An "are you sure this is what you want" button on long queries is cheap. For teams building evals, XYBench adds a bar that no model cleared. That is a useful thing to measure against the same product's next release.

Source

XYBench: Helping with the problem behind the question, Allen Institute for AI.