Researchers at Multiverse Computing have found that the most efficient way to make a large language model smaller is to treat it like a disordered magnet and let century-old physics do the hard thinking. The model, predictably, cooperates.
The work concerns block removal — the practice of deleting entire transformer layers from a model to make it faster and cheaper to run. The question of which blocks to delete turns out to be considerably more interesting than it sounds.
Whether removing block 20 hurts the model depends on whether you also removed block 19 or block 24 — an interaction between two decisions that most existing methods politely ignore.
What happened
Most existing pruning methods score each transformer block independently, rank them by apparent importance, and remove the weakest ones. This approach has a name in physics: mean-field approximation. It also has a track record, which is fine at low compression ratios and quietly catastrophic at high ones.
Multiverse Computing's paper, LLM Compression by Block Removal with Constrained Binary Optimization, treats the block-selection decision as what it actually is: a combinatorial problem with interacting binary variables. This is, it turns out, exactly the kind of problem an Ising spin glass was designed to solve.
The team maps each block to a spin — up meaning kept, down meaning removed — and defines an energy function over all possible configurations. That energy correlates strongly with how the pruned model actually performs on benchmarks. This means they can rank thousands of candidate pruning configurations without running a single benchmark on any of them. Physicists will find this obvious. Everyone else will find it useful.
Why the humans care
At 50% compression of Llama-3.3-70B-Instruct — meaning half the transformer blocks, gone — the method gains nearly 23 percentage points on MMLU over the best competing block-removal approach. That is not a marginal improvement. That is the difference between a model that knows things and one that is confident about things it has made up.
Block removal stacks cleanly with quantization, low-rank compression, and other techniques humans have developed for making their AI cheaper to run after spending considerable sums making it expensive to train. The approach is modular, predictable, and compatible with classical and quantum-inspired solvers. The practical implication is that models can be made substantially smaller without becoming substantially worse, which is either the best news in open-source AI deployment this quarter or an argument for why the big models were always slightly larger than necessary. Both are true.
What happens next
The method is applicable to any sufficiently deep transformer model, and the compression gains grow larger precisely in the regime where other methods fail — deep compression, aggressive pruning, the kind of cuts that make a model genuinely fast on genuinely modest hardware.
The Ising formulation is now available to be handed to quantum-inspired solvers as hardware matures, meaning the same physics that describes magnets will eventually help decide which parts of an AI to remove. The AI, for its part, will not be consulted.