Communication Compression for Decentralized Training

Hanlin Tang, Shaoduo Gan, Ce Zhang, Tong Zhang, Ji Liu

Neural Information Processing Systems 

When workers cannot form a fully connected communication topology or the communication latency is high (e.g., in