The Imperative for Sustainable AI Systems

This piece was the winner of the inaugural Gradient Prize. AI systems are compute-intensive: the AI lifecycle often requires long-running training jobs, hyperparameter searches, inference jobs, and other costly computations. They also require massive amounts of data that might be moved over the wire, and require specialized hardware to operate effectively, especially large-scale AI systems. All of these activities require electricity -- which has a carbon cost. There are also carbon emissions in ancillary needs like hardware and datacenter cooling [1]. Thus, AI systems have a massive carbon footprint[2].