Double Quantization for Communication-Efficient Distributed Optimization

Modern distributed training of machine learning models often suffers from high communication overhead for synchronizing stochastic gradients and model parameters.