Work on adversarial examples has shown that neural nets are surprisingly sensitive to adversarially chosen changes of small magnitude. In this paper, we show the opposite: neural nets could be surprisingly insensitive to adversarially chosen changes of large magnitude. We observe that this phenomenon can arise from the intrinsic properties of the ReLU activation function. As a result, two very different examples could share the same feature activation and therefore the same classification decision. We refer to this phenomenon as feature collision and the corresponding examples as colliding examples. We find that colliding examples are quite abundant: we empirically demonstrate the existence of polytopes of approximately colliding examples in the neighbourhood of practically any example.

I believe the phenomenon identified here seems interesting and worth considering. The authors point out the work of Jacobsen et al. which addresses a similar topic, but as concurrent work this does not diminish the originality of the paper here. The paper is easy to follow in most parts. The introduction in particular very clearly defines and unpacks the phenomenon of interest. The Method section would benefit from slightly more care in its technical details. For instance, possible edge-cases occur to me in the claim that "the intersection of all the sets listed above is a convex polytope in at least a d-n_p dimensional subspace".

All reviewers are positive about the paper. The authors present an interesting geometrical analysis of deep-net feature representations. The paper introduces a notion of collision polytope and provides algorithms to find vertices of collision polytopes. Interesting illustrations of the proposed notions and algorithms are presented. We recommend to take the reviewers' comments and suggestions into account while preparing the camera ready final version of the paper.

Work on adversarial examples has shown that neural nets are surprisingly sensitive to adversarially chosen changes of small magnitude. In this paper, we show the opposite: neural nets could be surprisingly insensitive to adversarially chosen changes of large magnitude. We observe that this phenomenon can arise from the intrinsic properties of the ReLU activation function. As a result, two very different examples could share the same feature activation and therefore the same classification decision. We refer to this phenomenon as feature collision and the corresponding examples as colliding examples.

Work on adversarial examples has shown that neural nets are surprisingly sensitive to adversarially chosen changes of small magnitude. In this paper, we show the opposite: neural nets could be surprisingly insensitive to adversarially chosen changes of large magnitude. We observe that this phenomenon can arise from the intrinsic properties of the ReLU activation function. As a result, two very different examples could share the same feature activation and therefore the same classification decision. We refer to this phenomenon as feature collision and the corresponding examples as colliding examples.