Anima Anandkumar: We have foundation models for language, not for physics

Summary

Anima Anandkumar argues that physics modeling (weather, fusion, fluid dynamics) cannot follow the scaling strategies of large language models (LLMs). Data scarcity and context lengths reaching trillions of points close off this path. Instead, the Neural Operators framework embeds physical structures, such as global harmonics, into the model. This allows for competitive predictions with far fewer examples.

Why it matters

If this claim is correct, it reframes a significant open question: Does the 'scaling law' based on scale and data generalize to continuous, multi-scale physical systems, or is inductive bias an unavoidable prerequisite? There are also practical implications. Million-fold speedups in weather forecasting and fusion plasma prediction on consumer GPUs could transform energy and climate infrastructure much faster than demanding simulations.

Highlights

• FourCastNet 3 achieves stable global weather projections months in advance by learning in the global harmonic frequency domain rather than on a raw grid.
• Fusion plasma disruption prediction works from just a few thousand examples and is approximately 1,000,000 times faster than traditional simulation.
• TorchLean embeds PyTorch-style networks into the Lean proof assistant. This enables formal verification of relevant bounds for safety-critical control loops.

Critical take

The article is not an independent analysis but a promotional podcast summary. It relays Anima's claims ('million times faster', 'all the computing power in the world would not be enough') without providing benchmarks, ablations, or failure modes. The 'scaling law' objection is also not proven by head-to-head comparisons with larger, unstructured foundation models; it is argued rhetorically. The reader is left to rely on the speaker's framing to distinguish between 'principled' approaches and 'mere scaling'.