MIT CSAIL details technique for shrinking neural networks without compromising accuracy

Deep neural nets are often quite large and require correspondingly large corpora, and training them can take days on even the priciest of purpose-built hardware. But it might not have to be that way. In a new study ("The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks") published by scientists at MIT's Computer Science and Artificial Intelligence Lab (CSAIL), deep neural networks are shown to contain subnets that are up to 10 times smaller than the entire network, but which are capable of being trained to make equally precise predictions, in some cases more quickly than the originals. The work is scheduled to be presented at the International Conference on Learning Representations (ICLR) in New Orleans, where it was named one of the conference's top two papers out of roughly 1,600 submissions. "If the initial network didn't have to be that big in the first place, why can't you just create one that's the right size at the beginning?" said PhD student and coauthor Jonathan Frankle in a statement.