Grad Init: Learning to Initialize Neural Networks for Stable and Efficient Training

Innovations in neural architectures have fostered significant breakthroughs in language modeling and computer vision.