The carbon footprint of a model can be complicated to determine and compare across modelling approaches and data centre infrastructures. A reasonable place to start may be by assessing the number of floating-point operations – that is, a discrete count of how many simple mathematical operations (for example, multiplication, division, addition, subtraction, and variable assignment) – that need to be performed to train a model. This factor and others can impact energy consumption along with the architecture of the model and the training resources, such as hardware like GPU or CPUs. Additionally, the physical considerations of the storage and cooling of the servers comes into play. As a final complication, it also matters where the energy is sourced from.
Sep-25-2021, 12:05:54 GMT