A Experimental Setup in Detail

We implement our attack framework using Python 3.7.3 and PyTorch 1.7.1 We run our experiments on a machine equipped with Intel i5-8400 2.80GHz 6-core processors, 16 GB of RAM, and four Nvidia GTX 1080 Ti GPUs. To compute the Hessian trace, we use a virtual machine equipped with Intel E5-2686v4 2.30GHz 8-core processors, 64 GB of RAM, and an Nvidia Tesla V100 GPU. For all our attacks in 4.1, 4.2, 4.3, and 4.5, we use symmetric quantization for the weights and asymmetric quantization for the activation--a default configuration in many deep learning frameworks supporting quantization. Quantization granularity is layer-wise for both the weights and activation. In 4.4 where we examine the transferability of our attacks, we use the same quantization granularity that the original studies describe [Choukroun et al., 2019, Zhao et al., 2019, Banner et al., 2019] while re-training clean models.