A Exponential Convergence

The exponential convergence can be proved for the two methods: expansion and sparse expansion. We first prove it for the expansion on sequential models, then generalize the result to more diverse architectures. Before detailing the proof of lemma A.1, we empirically motivate the assumption of symmetry over the weight values distribution. In Figure 4, we plot the distributions of the weights of several layers of a ResNet 50 trained on ImageNet. The assumption is often satisfied in practice. Furthermore, in any instances where it would not be satisfied, it can be enforced using asymmetric quantization.