Google DeepMind Researcher Says LLMs Cannot Generate New Ways of Thinking.

Summary

Google DeepMind researcher Tom Zahavy argues that large language models (LLM) can generate new outputs but cannot achieve the 'abductive leap' required to create entirely new conceptual frameworks.

Why it matters

This claim challenges the dominant narrative that AI is a genuine creator and underscores the limitations of current models in achieving radical innovation. By highlighting the need for a next-generation AI capable of reasoning beyond predefined rules, it has the potential to reshape how intelligent agents are developed.

Highlights

• Large language models (LLM) excel at recombining existing data but lack the ability to invent new premises or frameworks.
• Zahavy distinguishes deduction, induction, and abduction, noting that LLMs struggle with the abductive leap necessary for groundbreaking ideas.
• Future research could focus on 'thinking by doing' models that develop new concepts through interaction with the world, aligning with Yann LeCun's vision.

Critical take

While the argument is persuasive, it may underestimate emergent properties of large-scale models; these features can lead to unexpected conceptual breakthroughs without explicit 'leaps'.