Claude Fable 5 Captures 82% of Human Researchers in nanoGPT Benchmark

Summary

In Prime Intellect's August 2026 benchmark, 18 frontier models were compared against the human nanoGPT speed record (2,600 steps). Claude Fable 5 closed 82% of this gap with approximately 2,726 steps; however, no model produced a technique not found in the published literature.

Why it matters

This is one of the most comprehensive public evaluations of autonomous AI research to date. It clearly distinguishes how well current models perform in areas like synthesis, hypothesis-driven experimentation, and implementation from what they still cannot do—generating truly novel ideas. The finding of zero innovation caps the impressive 82% figure and reframes the 'AI researcher' narrative around a measurable, concrete ceiling.

Highlights

• Claude Fable 5 reached approximately 2,726 steps, surpassing the 2,600-step human record. The model outperformed Opus 5 (54%) and GPT-5.5 (8%).
• None of the 18 models offered a new method beyond techniques already present in the published literature.
• The most successful models stood out due to hypothesis-driven experimental design, seed noise management, and the calibrated re-evaluation of previously failed results.

Critical take

The phrase '82% of the gap' may be misleading. The human record was established over months under conditions including internet access, forum discussions, and the ability to rest and reflect. These conditions were explicitly withheld from the models; therefore, the comparison is not apples-to-apples. Additionally, the article weakens its analytical focus by sprinkling promotional references to OpenAI Presence and Claude Code throughout the text. This reads more like sponsored content than a rigorous benchmark review.