Minibatch vs Local SGD for Heterogeneous Distributed Learning

Neural Information Processing Systems 

Local SGD that improves over Minibatch SGD in a non-homogeneous regime.