Inference Scaling Reshapes AI Governance

The shift from scaling up the pre - training compute of AI systems to scaling up the ir inference compute may have profound effects on AI governance. The nature of these effects depends crucially on whether this new inference compute will primarily be used during external deployment or as part of a more complex training programme within the lab. R apid scaling of inference - at - deployment would: lower the importance of open - weight models (and of securing the weights of closed models), reduce the impact of the first human - level models, change the business model for frontier AI, reduce the need for power - intense data centres, and derail the current paradigm of AI governance via training compute thresholds. R apid scaling of inference - during - training would have more ambiguous effects that range from a revitalisation of pre - training scaling to a form of recursive self - improvement via iterated distillation and amplification . The intense year - on - year scaling up of AI training runs has been one of the most dramatic and stable markers of the Large Language Model era . Indeed it had been widely taken to be a permanent fixture of the AI landscape and the basis of many approaches to AI governance. But recent reports from unnamed employees at the leading labs suggest that their attempts to scale up pre - training substantially beyond the size of GPT - 4 have led to only modest gains which are insufficient to justify continuing such scaling and perhaps even insufficient to warrant public deployment of th o se models ( Hu & Tong, 2024) . A possible reason is that they are running out of high - quality training data. While the scaling laws might still be operating (given sufficient compute and data, the models would keep improving), the ability to harness them through rapid scaling of pre - training may not.