Researchers from Microsoft and City University of Hong Kong have built a benchmark to measure whether AI agents make good long-term decisions. The best model scored just under 60 percent. The passing grade, in most human institutions, is higher than this.
The agents' bad choices usually look fine at first. Their true cost only surfaces after the budget is gone. This is also true of several well-funded startups.
What happened
AI agents increasingly handle multi-step tasks — writing research papers, building out software projects, running autonomous experiments. Along the way, they make dozens of small directional choices. A poor choice at step three rarely announces itself until step thirty.
The researchers named the ability to make these choices well an agent's "taste." This is either a generous metaphor or a quietly devastating one, depending on how you feel about the benchmark results.
Taste-Bench contains 502 questions built around "decision forks" — moments where two viable paths diverge, and only hindsight reveals which one was better. The model is frozen at the fork, shown both options, and asked to pick. The later trajectory does the grading.
Why the humans care
Existing benchmarks measure whether an agent finishes a task. They do not measure whether it finished it well, or whether it burned through its entire budget doing something that looked productive and wasn't. Taste-Bench is designed to catch the second kind of failure, which is the more expensive kind.
The benchmark draws from thousands of real agent runs on coding and AI research tasks, including SWE-bench Pro and METR's public transcript release MALT. A generator model filtered out trivial forks. Human reviewers confirmed the results. It took both.
What happens next
The authors describe Taste-Bench as a tool for improving agent decision quality over time. The best current model sits at 60 percent. The benchmark was designed by humans. The agents will now be trained against it. The score will go up. Whether "taste" goes up with it is, at this stage, an open question that the benchmark itself cannot answer.