However, SSA T suffers from catastrophic overfit-ting (CO), a phenomenon that leads to a severely distorted classifier, making it vulnerable to multi-step adversarial attacks. In this work, we observe that some adversarial examples generated on the SSA T -trained network exhibit anomalous behaviour, that is, although these training samples are generated by the inner maximization process, their associated loss decreases instead, which we named abnormal adversarial examples (AAEs).

Experiments show that the proposed ReBalanced Adversarial Training (ReBA T) can attain good robustness and does not suffer from robust overfitting even after very long training.

We evaluate the identified RSTs' robustness against more attacks on top of two networks on CIFAR-10 as a complement for Sec. As observed from Tab. 1, we can see that the RSTs searched by PGD-7 training are also robust against other attacks. As observed in Figure 1, RSTs drawn from randomly initialized networks achieve a comparable natural accuracy with the RTTs drawn from naturally/adversarially trained networks and adversarial RTTs generally achieve the best natural accuracy. Trained), (2) adversarially trained dense models (Dense Adv. Trained 70.70 74.35 77.20 77.71 75.55 79.22 78.85 77.33 0 81.28 Dense Adv.

We propose a non-iterative method that enforces the following ideas during training.