Learning with Hierarchical Gaussian Kernels

Although kernel methods such as support vector machines are one of the state-of-the-art methods when it comes to fully automated learning, see e.g. the recent independent comparison [7], the recent years have shown that on complex datasets such as image, speech and video data, they clearly fall short compared to deep neural networks. One possible explanation for this superior behavior is certainly their deep architecture that makes it possible to represent highly complex functions with relatively few parameters. In particular, it is possible to amplify or suppress certain dimensions or features of the input data, or to combine features to new, more abstract features. Compared to this, standard kernels such as the popular Gaussian kernels simply treat every feature equally. In addition, most users of kernel machines probably stick to the very few standard kernels, often simply because there is in most cases no principled way for finding problem specific kernels.