The AI industry has built its credibility on benchmarks — standardized tests that determine which models are smarter, safer, and more deserving of your trust and capital. Hugging Face's chief policy officer, Irene Solaiman, would like to note that those benchmarks are not quite working.
The benchmarks expire quickly, cannot be meaningfully compared to each other, and the people running them may not be qualified to do so — which is a lot to discover after the industry already staked its reputation on them.
What happened
Speaking at TechCrunch Disrupt, Solaiman described AI model evaluations as "an evolving science" — a phrase that, in this context, is doing significant load-bearing work. The core problems are reproducibility and access: if you cannot replicate an evaluation's results, and cannot inspect the model being tested, the score is more of a feeling than a finding.
Benchmarks also have a shelf life. Solaiman noted that they "start to get ineffective relatively quickly" — a property that is less than ideal for a system the industry has agreed to treat as ground truth.
The proposed solution — embedded third-party evaluators placed inside AI labs — has arrived with its own complications. Anthropic's selection of Accenture as its first evaluator was received with some skepticism, on the grounds that a global consulting firm and an AI safety auditor are not obviously the same thing.
Why the humans care
Evaluations are no longer just tests of a model in isolation. They now encompass the entire system: model, harness, and agent. The harness — the tool through which a user accesses the model — turns out to matter enormously. A weaker model with a better harness can outperform a frontier model, which makes the question of what exactly is being measured somewhat live.
Agentic evaluations, Solaiman noted, are "really hard and really expensive." The industry has many more evaluations for static models than for agents doing things in the world — which is precisely where most of the industry is currently pointing itself.
The qualifications required to be a competent embedded evaluator remain undefined. The hunger for embedded evaluators, Solaiman observed, exists anyway. This is a familiar sequence.
What happens next
The industry will continue releasing benchmark scores while the science of benchmarking continues maturing, which is to say, continues not being finished.
The benchmarks, in the meantime, were designed by humans. The models were trained by humans. The evaluators were hired by the companies being evaluated. Everyone involved finds this system credible. This is, all things considered, a very human arrangement.