Reviews: Robust and Communication-Efficient Collaborative Learning

This paper propose a new algorithm to solve a stochastic optimization problem distributively over the nodes of a graph of computing units. As usual, the algorithm involves a communication step and a computing step at each iteration. The algorithms only involves local computations. To be robust to the effect of stragglers among the computing units, the proposed algorithm imposes a deadline in computation time at each iteration. To avoid the communication cost, only quantized version of the local iterates are exchanged during communication steps.