Shrinking deep learning's carbon footprint

In June, OpenAI unveiled the largest language model in the world, a text-generating tool called GPT-3 that can write creative fiction, translate legalese into plain English, and answer obscure trivia questions. It's the latest feat of intelligence achieved by deep learning, a machine learning method patterned after the way neurons in the brain process and store information. But it came at a hefty price: at least $4.6 million and 355 years in computing time, assuming the model was trained on a standard neural network chip, or GPU. The model's colossal size -- 1,000 times larger than a typical language model -- is the main factor in its high cost. "You have to throw a lot more computation at something to get a little improvement in performance," says Neil Thompson, an MIT researcher who has tracked deep learning's unquenchable thirst for computing.