We thank all the Reviewers for a careful reading of our paper and for providing useful suggestions for improvements, 1 which we will be happy to implement in the camera-ready version

As we state at the beginning of Sec. 2, Theorems 1 and 3 hold for any activation function, including We will clarify this point in the camera-ready version. Figure 1: Histogram of correctly and incorrectly classified pictures shows that trained neural networks are far more likely to misclassify points closer to a classification boundary for both the training and test sets. Results are aggregated across 20 different trained neural networks. We will move the MNIST results to the main paper swapping them with the detailed proofs and modify Sec. We will add in the camera-ready version a discussion on the convergence rate to the Gaussian probability distribution.