What Is Weight Sharing In Deep Learning And Why Is It Important

NAS, however, is computationally expensive for automating and democratising machine learning. The initial success of NAS was attributed partially to the weight-sharing method, which helped in the dramatic acceleration of probing the architectures. But why is the weight sharing method being criticised? Traditionally, NAS methods were expensive due to the combinatorially large search space, requiring to train thousands of neural networks to completion. In 2018, ENAS (Efficient NAS) paper, introduced the idea of weight-sharing, in which only one shared set of model parameters is trained for all architectures.