Response to Reviewer # 1

The goal of this paper is to determine whether neural networks (NN) are equivalent to kernel methods or not. In this paper, we argue in the opposite direction, namely NN is superior to NT. All of these generalizations are accessible to the same proof techniques developed in our paper. Before our paper, this problem was open for quadratic activations as well. We have extensive experimental results comparing NN and NT for RELU and Tanh activations (see figure).