TheSurprisingSimplicityoftheEarly-TimeLearning DynamicsofNeuralNetworks

Neural Information Processing Systems 

In this work, we show that these common perceptions can be completely false in the early phase of learning.