Spectral Analysis of Kernel and Neural Embeddings: Optimization and Generalization

Kernel methods are one of by a spectral analysis of representations corresponding the pillars of machine learning, as they give us a flexible to kernel and neural embeddings. They framework to model complex functional relationships in a showed that in a simple single layer network, the principled way and also come with well-established statistical alignment of the labels to the eigenvectors of the properties and theoretical guarantees. The interplay of corresponding Gram matrix determines both the kernels and data labellings has been addressed before, for convergence of the optimization during training example in the work on kernel-target alignment (Cristianini as well as the generalization properties. We show et al., 2001). Recently, (Belkin et al., 2018) also make the quantitatively that kernel and neural representations case that progress on understanding deep learning is unlikely improve both optimization and generalization.