A.1 Theoretical Proof The following is proof for Theorem 1 and 2 on Upper Bound on Lipschitz Constant of a DNN with Gaussian Distributed Weights, which is inspired by [67-69]. Let A be an (N n) matrix whose elements are independent standard normal random variables. Then, N n E[λmin(A)] E[λmax(A)] N+ n, where λmin and λmax denote the minimum and maximum singular values of A, respectively, and E[ ] represents the expected value. This can be extended to convolutional neural networks (CNN). Using doubly block circulant matrix the convolution operation can be represented by matrix multiplication.

By forcing at most N out of M consecutive weights to be non-zero, the recent N:M network sparsity has received increasing attention for its two attractive advantages: 1) Promising performance at a high sparsity.

For the synthetic-to-real domain generalization (DG), we use one of the synthetic datasets (GTAV [12] or SYNTHIA [13]) as the source domain and evaluate the model performance on three real-world datasets (CityScapes [2], BDD-100K [16], and Mapillary [11]). GTAV [12] contains 24,966 images with the size of 1914 1052. It is splited into 12,403, 6,382, and 6,181 images for training, validating, and testing. SYNTHIA [13] contains 9,400 images of 960 720, where 6,580 images are used for training. We use the validation sets of the three real-world datasets for evaluation.

But could more performance be squeezed out of networks by using different input representations?

But these methods are unable to improve throughput (frames-per-second) on real-life hardware while simultaneously preserving robustness toadversarial perturbations.