SBO-RNN: Reformulating Recurrent Neural Networks via Stochastic Bilevel Optimization

Neural Information Processing Systems 

We prove that under mild conditions there is no vanishing or exploding gradient in training SBO-RNN.