Reviews: On Exact Computation with an Infinitely Wide Neural Net

This paper has two main contributions. First, the convolutional extension of the neural tangent kernel (CNTK) was proposed and then an algorithm using CNTKs for "exact computation with an infinitely wide neural net" was designed. The algorithm allows squared-loss kernel regression with CNTKs corresponding to infinitely wide vanilla CNNs with ReLU activation as well as those also with global average pooling. Its time complexity is linear in the depth and quadratic in the amount of data and the height and width of the images. This time complexity, in previous papers, was believed to be impossible.