Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback

Shuai Zheng, Ziyue Huang, James Kwok

Neural Information Processing Systems 

Communication overhead is a major bottleneck hampering the scalability of distributed machine learning systems.