Adversarially robust classification, that has been of much recent interest, is typically formulated as follows.

Our study begins with an observation that the BW metric has alinear dependence on SPD matrices in contrast tothe quadratic dependence ofthe AI metric.

The Information Bottleneck (IB) objective uses information theory to formulate a task-performance versus robustness trade-off.

The details include: Searching in the searched space. Q-K -V dimension could be smaller than the embedding dimension. In this section, we present the details of supernet training and evolutionary algorithm. At last, we update the corresponding weights with the fused gradients. Alg. 2 shows the evolution search in our method.

In this supplementary material, we provide the proof of all theoretical results stated in the paper. We complete the proof in 8 steps by showing statements 1 - 8 above. Markov's inequality the first term on the RHS is also O This follows from Step 8 and the fact that by Chebyshev's inequality, The reasoning here is similar to Step 1. Since the number of splits K is bounded, we only need to verify for any k { 1, 2,...,K }, null null null null 1 n null Below we'll prove 1 n null Combining the above, we obtain (30). In the last inequality we utilize (32).