Reviews: Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization
–Neural Information Processing Systems
The paper tightens the analysis of local SGD that periodically averages the models at different nodes. It improves the bound on the communication rounds suffice to achieve linear speedups and relaxes some of the assumptions used in previous works. The authors also develop an adaptive scheme to choose the communication rounds based on the intuition from their theoretical results. They also provided empirical results on logistic regression problem with epsilon dataset to support the theoretical results. Comments: The paper is well written and east to follow.
Neural Information Processing Systems
Jan-26-2025, 21:04:21 GMT
- Technology: