Deep Learning Has Hit a Wall, Intel's Rao Says

The rapid growth in the size of neural networks is outpacing the ability of hardware to keep up, said Naveen Rao, vice president and general manager of Intel's AI Products Group, at the company's AI Summit yesterday. Solving the problem will require rethinking how processing, network, and memory work together, he said. "Over the last 20 years we've gotten a lot better at storing data," Rao said during a one-hour presentation at Intel's AI Summit 2019 in San Francisco Tuesday. "We have bigger data sets than ever before. Moore's Law has led to much greater compute capability in a single place. And that allowed us to build better and bigger…neural network models. This is kind of a virtuous cycle and it's opened up new capabilities."