Simplest example that SVMs can't handle, but neural nets can? • /r/MachineLearning

Yes, "finding the right kernel" and "finding the right features" are really the same problem in the sense that the kernel matrix is the Gram matrix of your data represented in the feature space of the kernel. In the "neural network-y" approach you think about the "primal" problem and design the feature space where the dot products happen, while leaving the dot product implicit. Given a feature space you can write down the kernel explicitly if you want (like I did in my previous post), but usually you don't need to do this if you're living in the neural network world. The "kernel-y" approach is to think about the "dual" problem where you design the dot product, while leaving the feature space where the dot products happen implicit. Given a kernel you can write down the feature space explicitly if you want (depending on the kernel this can be somewhat involved), but usually you don't need to do this if you're living in the kernel world.