Green AI

Since 2012, the field of artificial intelligence (AI) has reported remarkable progress on a broad range of capabilities including object recognition, game playing, speech recognition, and machine translation.43 Much of this progress has been achieved by increasingly large and computationally intensive deep learning models.a Figure 1, reproduced from Amodei et al.,2 plots training cost increase over time for state-of-the-art deep learning models starting with AlexNet in 201224 to AlphaZero in 2017.45 The chart shows an overall increase of 300,000x, with training cost doubling every few months. An important paper47 has estimated the carbon footprint of several NLP models and argued this trend is both environmentally unfriendly and prohibitively expensive, raising barriers to participation in NLP research. We refer to such work as Red AI. The amount of compute used to train deep learning models has increased 300,000x in six years. Figure taken from Amodei et al.2 This trend is driven by the strong focus of the AI community on obtaining "state-of-the-art" results,b as exemplified by the popularity of leaderboards,53,54 which typically report accuracy (or other similar measures) but omit any mention of cost or efficiency (see, for example, leaderboards.allenai.org).c Despite the clear benefits of improving model accuracy, the focus on this single metric ignores the economic, environmental, and social cost of reaching the reported results.