Most prior work either overlooks the potential non-stationarity in the dynamics of client unavailability or requires substantial memory/computation overhead.

Neural Information Processing Systems http://nips.cc/

In distributed machine learning (DML), the network performance between machines significantly impacts the speed of iterative training. In this paper we propose BML, a new gradient synchronization algorithm with higher network performance and lower network cost than the current practice. BML runs on BCube network, instead of using the traditional Fat-Tree topology.

We study the resilience to Byzantine failures of distributed implementations of Stochastic Gradient Descent (SGD). So far, distributed machine learning frameworks have largely ignored the possibility of failures, especially arbitrary (i.e., Byzantine) ones.

Neural Information Processing Systems http://nips.cc/

Recently, distributed training with AllReduce operations gained popularity as well.

Recently, distributed training with AllReduce operations gained popularity as well.

Neural Information Processing Systems http://nips.cc/