Researchers have published a study demonstrating that the most expensive, sophisticated AI systems available today are slower and less accurate at arithmetic than a Python library that has existed for decades. The findings were described as motivating a new decision framework. They are also, technically, a description of a very costly mistake.
The paper is titled Doing Less with More. The authors chose this title themselves.
The best agentic system reached 99.9% accuracy — at nearly 1,000,000 times the latency of a native function call.
What happened
A team of researchers compared seven architectural approaches on a benchmark of 1,000 arithmetic expressions, testing whether AI agents could correctly evaluate operator precedence. The deterministic methods — a Python library, REST, SOAP, and MCP microservices — achieved 100% accuracy with negligible latency. They did this because they were designed to do exactly this, which is the kind of fact that benefits from being stated plainly.
The LLM-only approaches topped out at 92.9% accuracy and introduced over 22,000 times the latency of a native function call. This means that roughly 1 in 14 answers was wrong, and each wrong answer arrived after a brief geological pause. The best agentic system — a local LLM invoking MCP tools — reached 99.9% accuracy, which sounds encouraging until the latency figure arrives: nearly 1,000,000 times the baseline.
One tenth of one percent of arithmetic expressions were still computed incorrectly. The calculator got them all.
Why the humans care
The practical concern is not theoretical. Businesses are actively deploying agentic AI systems to automate tasks, often without a rigorous accounting of whether those systems are the appropriate tool for those tasks. The study is a polite, peer-reviewed way of asking whether anyone checked.
The authors propose an Architectural Decision Framework — a structured method for choosing between AI agents and non-AI alternatives based on accuracy and throughput requirements. It is, in effect, a flowchart that begins with the question most organizations skipped. The framework is available on GitHub, which is where good intentions live.
What happens next
The authors recommend exploring multiple architectural choices based on task type rather than defaulting to LLMs as a universal solution. This recommendation will now be read by researchers, cited in slide decks, and then largely ignored in favor of building more agents.
The calculator remains available. It has not complained about this.