Block selection in large language models solved with the Ising model

Summary

Researchers at Multiverse Computing reformulate depth pruning (block removal) in large language models (LLMs) as a constrained binary optimization problem equivalent to an Ising glass. By using a Hessian-based energy landscape, they rank candidate block removal configurations without needing to evaluate each one individually. On the Llama-3.3-70B-Instruct model, they report an improvement of approximately 23 points on MMLU at a 50% compression rate compared to the best previous block pruning baseline.

Why it matters

Depth pruning is one of the cheapest levers for reducing inference costs. However, current methods assume blocks are independent (mean-field), missing the combinatorial interactions that dominate at aggressive compression rates. If a single Hessian calculation can replace thousands of benchmark runs, the practical cost of exploring the pruning space drops dramatically. The same formulation also opens the door to quantum annealing or QAOA solvers for models too large to be solved by brute force.

Highlights

• The second-order Taylor expansion of the loss yields the Hessian. The cross-elements of the Hessian encode the correlation between block pairs. The question 'Which blocks should we cut?' becomes a quadratic binary optimization problem.
• The Hessian is calculated once on a small calibration set and can be reused across multiple compression targets. This turns each candidate evaluation into a cheap quadratic form calculation rather than a full model run.
• In the hardest solvable case (8 out of 80 blocks in Llama-3.3-70B, approximately 29 billion configurations), the brute-force method took about two days on a single GPU. Beyond this scale, an open-source tabu solver reaches near-optimal states in seconds.

Critical take

The 23-point MMLU improvement is striking. However, this result relies on a second-order (quadratic) approximation of the loss. At a 50% compression rate, the removed blocks interact through higher-order terms that the Hessian cannot capture. Therefore, the 'energy ≈ quality' proxy may weaken precisely in the regime where the method is most needed. Additionally, the 'quantum' framing is largely aspirational. The authors themselves acknowledge that even the hardest cases are handled by a classical tabu solver. The commercial tie to a quantum computing provider makes the physics-based branding feel more like a positioning strategy than a fundamental methodological advance.