Appendix A Margin Bound A.1 Toy Example Let f

Section 3.2, as we trace down the associated inequality bound in Recall that we have defined in Section 3.1 the notion of output Using the notation in Section 3.1, we have In the following proof for Theorem 2, we apply similar steps in Appendix A.2 and consider the difference between set of pairwise margin under natural and weight perturbation setting, recall in We now offer a similar proof for convolutional neural networks. We note that each convolution operation can be described as matrix multiplication of a doubly block Toeplitz matrix. We now provide a proof for Lemma 2. Recall the definition of ramp function in Section 3.4.1, We further consider the case of cross entropy and prove an upper bound for it. We've also conducted the experiment of convolution-layer based model training on CIFAR-10 using As shown in Table 4, the standard model's performance rapidly degrades in 5-folds when the perturbation radius is The experiment setting follows Figure 1(b).