Reviews: Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients

Neural Information Processing Systems 

The paper extends the lazily aggregated gradient (LAG) approach by applying quantization to further reduce communication. In the original LAG approach, workers only communicate their gradient with the central coordinator if it is significantly different from its previous one. In this paper, the gradients are compressed using quantization and workers skip communication if their quantized gradient does not differ substantially from previous ones. For strongly convex objectives, the paper proves linear convergence. The paper is very well written and the approach is clearly motivated, easy to understand, and discussed in the context of related work.