On Linear Stability of SGD and Input-Smoothness of Neural Networks

Neural Information Processing Systems 

This is relevant when the learning rate is not very small.