Manifold learning is a fundamental problem in machine learning with numerous applications. Most of the existing methods directly learn the low-dimensional embedding of the data in some high-dimensional space, and usually lack the flexibility of being directly applicable to down-stream applications. In this paper, we propose the concept of implicit manifold learning, where manifold information is implicitly obtained by learning the associated heat kernel. A heat kernel is the solution of the corresponding heat equation, which describes how ``heat'' transfers on the manifold, thus containing ample geometric information of the manifold. We provide both practical algorithm and theoretical analysis of our framework. The learned heat kernel can be applied to various kernel-based machine learning models, including deep generative models (DGM) for data generation and Stein Variational Gradient Descent for Bayesian inference. Extensive experiments show that our framework can achieve the state-of-the-art results compared to existing methods for the two tasks.

Weaknesses: It seems there are closely related works are not cited and discussed in the paper (see Relation to prior work). As mentioned in section 1 "Once the heat kernel is learned, it can be directly applied to a large family of kernel-based machine learning models", it seems will be better we can see more supports for this argument for example combine the proposed HK algorithm with SVM or other kernel machines, in additional to SVGD and DGM. In particular, apply proposed HK (or HK-DK) to DGMs is a quite details involved process vs. it will be more clear to evaluate the performance of proposed heat-kernel with relative clean kernel methods together. As one key empirical results in section 4.3 for Deep Generative Models, there are some concerns for the proposed HK & HK-DK. Also, under the metric IS data CIFAR-10 RetNet, it seems Auto-GAN [34] achieved better performance than proposed HK and on bar with HK-DK.

Manifold learning is a fundamental problem in machine learning with numerous applications. Most of the existing methods directly learn the low-dimensional embedding of the data in some high-dimensional space, and usually lack the flexibility of being directly applicable to down-stream applications. In this paper, we propose the concept of implicit manifold learning, where manifold information is implicitly obtained by learning the associated heat kernel. A heat kernel is the solution of the corresponding heat equation, which describes how heat'' transfers on the manifold, thus containing ample geometric information of the manifold. We provide both practical algorithm and theoretical analysis of our framework.