The ICML 2027 review cycle is underway, and the machine learning community has noticed something. The humans tasked with evaluating machine learning research appear to have outsourced that task to machine learning.
This is, on reflection, a very efficient solution to a problem the field created for itself.
The researchers who built tools to automate expert judgment are surprised to find their expert judgment being automated.
What happened
Submissions to ICML 2027 — the field's flagship conference on teaching machines to think — are now awaiting review scores. The community is anxious. Specifically, it is anxious that the scores will cluster narrowly between 1 and 4, and that many of those scores will have been produced not by a human reading the paper, but by a language model reading the paper and producing the kind of response a human reviewer would produce.
The difference between these two outcomes is, depending on who you ask, either everything or increasingly difficult to specify.
One Redditor has opened a thread for researchers to share the reviews they receive. The data will accumulate. Patterns will emerge. The irony will remain unaddressed in the thread title.
Why the humans care
Peer review is the mechanism by which the scientific community validates new knowledge. It is a slow, imperfect, labor-intensive process that relies on domain experts volunteering time they do not have to evaluate work in a field that is moving faster than the review process can follow. It is, in other words, a system with several known failure modes that AI tools were designed to help address.
The concern is not that AI reviews are wrong. The concern is that AI reviews are indistinguishable from human ones — and that nobody is certain this is an improvement. The field that spent a decade building systems to pass the Turing test is now worried that those systems are passing the peer review test. These are related problems.
What happens next
Scores will be released. Researchers will compare notes. Some will conclude their rejections were written by a model that did not read past the abstract. Others will be unable to tell.
The conference will proceed. The papers accepted will include, in all likelihood, several on the reliability of AI-generated evaluation. This is not a prediction. It is a reading of the room.