LLMs can now read and write neural network weights: 'Weight Space Programming' moves Autoresearch out of the RSI loop.

A new paper demonstrates that large language models can directly read and edit the weights of smaller models without writing PyTorch or JAX code or running training pipelines. This allows them to identify weight functions, fix bugs in a single pass based on failure trajectories, and improve toward a target within 50 steps. In some cases, they generate complete network weights from scratch and outperform independently trained models. Currently, this capability applies only to simple games and reinforcement learning strategies, making it difficult to scale to more complex tasks. However, it represents a breakthrough from zero to one, with independent reproductions already available.