On Sampling Strategies for Spectral Model Sharding

Neural Information Processing Systems 

The general recipe for each communication round is composed of three steps.