Discovering cryptographic weaknesses with Claude

Summary

Anthropic researchers used Claude Mythos to uncover mathematical errors in the HAWK encryption algorithm and a weakened AES variant; this process required extensive prompting over a 60-hour runtime.

Why it matters

This work demonstrates that large language models can assist in cryptanalysis and have the potential to accelerate the discovery of subtle algorithmic weaknesses. Additionally, by collaborating with leading universities, they are establishing a new benchmark called CryptanalysisBench to systematically evaluate the cryptanalytic capabilities of large language models.

Highlights

• Claude Mythos identified non-trivial flaws in HAWK and a weaker AES variant.
• The process required intensive prompting and incentives; approximately $100,000 in API costs were incurred over 60 hours.
• In partnership with ETH Zurich and other institutions, a new evaluation framework called CryptanalysisBench was created; this framework aims to measure the cryptanalytic performance of large language models.

Critical take

While impressive, the findings have no immediate practical impact on existing systems and rely heavily on costly, manually guided prompting; this raises questions about scalability and reproducibility.