The Model Is the Byproduct

Last Friday, Andrej Karpathy open-sourced a 630-line Python script and went to bed. By morning, an AI agent running on a single GPU had completed roughly 100 complete LLM training runs, each lasting exactly five minutes, autonomously modifying the neural network architecture, the optimizer, the hyperparameters, evaluating the results, keeping improvements, discarding failures, and moving on to the next experiment. No foundation model. No API calls to a frontier lab. Just data, a training loop, a