Neural networks are sensitive to architecture and hyper-parameter choices [3,4]. For example, on the ImageNet dataset [5], we have observed a large increase in accuracy thanks to changes in architectures, hyper-parameters, and training algorithms, from the seminal work of AlexNet [5] to recent state-of-the-art models such as EfficientNet [6]. However, as neural networks become increasingly complex,thepotential number ofarchitecture and hyper-parameter choices becomes numerous.

While highly efficient, themethod exploits only a coarse approximation to the loss landscape and can easily fail when a small value is required. Aiming at more powerful attacks, I-FGSM [28] and PGD [32] are further introduced to generate adversarial examples inaniterativemanner.

Such compression problems naturally arise in distributed and federated learning. We provide novel mathematical results and derivecomputationally efficient algorithms thataremore accurate than previous compression techniques.