OnScramblingPhenomena forRandomlyInitializedRecurrentNetworks

Neural Information Processing Systems 

Recurrent Neural Networks (RNNs) frequently exhibit complicated dynamics, and their sensitivity to the initialization process often renders them notoriously hardtotrain.