Qwen 3.8 27B can interpret neural network weights

Summary

Simon Willison reproduced the 'Weight-Space Programming' experiment locally. The experiment demonstrated that the Qwen 3.8 27B model, running on a laptop, could identify which simplified grid game (Pac-Man, Pong, or Breakout) a small reinforcement learning network was trained to play by reading its raw floating-point weights. However, the model failed on the less common Freeway task.

Why it matters

This proves that the ability to interpret neural network weights without executing code is no longer exclusive to frontier proprietary models like GPT-6 Astra. Consequently, interpretability via inspection is becoming a practical local capability. However, Willison provides a limiting framework for this enthusiasm. The task is not general model surgery; it is a narrow recognition puzzle on toy-sized 8x8/10x10 networks. Furthermore, the reasoning traces are not transparent even to the author.

Highlights

• Qwen 3.8 27B (17 GB, local) correctly identified 3 out of 4 games from raw weights; its self-reported confidence ranged from 72% to 88%
• The article conflates the concepts of 'tool-free weight reading' and 'weight modification via code execution.' Willison argues that these two situations should be treated separately
• The reasoning traces are at a level the author himself finds 'almost incomprehensible.' It seems reasonable that a competent human with pen and paper could reach the same conclusion

Critical take

The experiment is genuinely charming; however, it is far from the 'end of source code' narrative some commentators advocate. This is a multiple-choice recognition task on four toy Atari-style networks, not open-ended weight editing. Willison himself notes that the Qwen3.5-0.8B 'correction' example in the article reflects steps any human machine learning engineer would take. The confidence scores in the 72%–88% range and the error on Freeway emphasize that this is fragile pattern matching, not true mechanical understanding.