How Autoresearch will change Small Language Models adoption

Autoresearch is a tool that ingests a training script and a metric and then edits code, runs experiments, checks if the metric improved, keeps or discards, and repeats. It allows researchers to optimize models overnight, even without any machine learning experience. Andrej Karpathy used it to squeeze 11% more speed out of his GPT-2 training. Shopify's CEO used the tool to train a 0.8B model overnight that outscored his previous 1.6B model.