Appendix A Related works In this section, we review closely related literature on decentralized optimization, communication-efficient algorithms, and communication compression

In this section, we review closely related literature on decentralized optimization, communication-efficient algorithms, and communication compression. Both categories have received significant attention in recent years. Here, (7) is referred as the Cauchy-Schwarz inequality, (8) and (9) are referred as Y oung's inequality. Specifically, we have the following lemma. Proof: We first prove (11) by induction.