MARCH scales recurrent memory using content-routed state anchors

Summary

MARCH introduces a recurring architecture that periodically caches state checkpoints as content-guided anchors, enabling scalable long-range memory without the quadratic cost of Transformers.

Why it matters

The model bridges the gap between efficient computation and high retrieval performance, offering a controllable trade-off between historical resolution and memory cost. This could reshape how large language models handle long contexts, especially in resource-constrained environments.

Highlights

• Periodic caching of recurrent state checkpoints as content-conditioned anchors.
• Attention-mechanism style merging over all anchors that are causally accessible at each token.
• Consistent outperformance of linear attention variants on LongBench and in-context retrieval tasks.

Critical take

While MARCH alleviates quadratic complexity, the growing memory bank can still impose significant storage and retrieval overhead for very long sequences; this may limit its practical scalability.