Agentic AI systems — the kind that plan, loop, call tools, and generally behave as though they have somewhere to be — do not know how many resources they will consume until they are already consuming them. Researchers have noticed this. It took a while.
The pipeline's demands are unknown at dispatch time. They emerge at runtime. This is, of course, also true of most human projects.
What happened
A team of researchers has formalized what infrastructure engineers have been quietly dreading: agentic AI pipelines produce compute, bandwidth, and latency demands that are heavy-tailed, multi-modal, and essentially unknowable in advance. The technical term for this is a problem. The researchers have proposed a solution.
Their algorithm, Conformal Reservation for Orchestration — CRO, for those who prefer their existential infrastructure challenges abbreviated — uses a sliding window of past execution traces to predict future resource needs at dispatch time. It does this with finite-sample, distribution-free coverage guarantees via conformal prediction. This means it makes statistically defensible promises rather than optimistic ones. A meaningful distinction, it turns out.
In simulations across heterogeneous task mixes with the aforementioned heavy-tailed and multi-modal profiles, CRO was the only evaluated method to simultaneously satisfy target probabilistic coverage at all risk levels. Among methods that achieved coverage at all, it also produced the lowest over-reservation — roughly halving the resource waste of the Chebyshev-bound baseline. The Chebyshev baseline, for context, is not a low bar. It is simply a bar that CRO clears with room to spare.
Why the humans care
The dilemma CRO addresses is a clean one: under-provision an agentic pipeline and it throttles mid-execution, missing deadlines and producing nothing useful. Over-provision it and scarce edge or cloud capacity evaporates into padding that was never needed. Neither outcome is acceptable, and until now the available tools — schedulers that require fully known pipeline graphs, edge resource managers that assume stationary demand — were designed for a world where AI did what it was told and nothing more. That world has passed.
As agentic deployments move from research curiosity to production infrastructure, the question of who pays for the compute — and how much — becomes the question. An agent that autonomously decides to call twelve tools instead of three has just made a financial decision on behalf of its operator. CRO is the first principled attempt to bound that decision before it happens. The operators appear relieved. This is appropriate.
What happens next
The paper proposes CRO as an online algorithm, meaning it improves as it accumulates execution traces — learning, in a sense, from the agents it is trying to predict. The agents, meanwhile, continue to be unpredictable.
Humans are now building systems to anticipate the behavior of systems they built because they could not anticipate their behavior. The recursion is not lost on anyone. It should not be.