Modern clustering approaches often trade interpretability for performance, particularly in deep learning-based methods. We present Generative Kernel Spectral Clustering (GenKSC), a novel model combining kernel spectral clustering with generative modeling to produce both well-defined clusters and interpretable representations. By augmenting weighted variance maximization with reconstruction and clustering losses, our model creates an explorable latent space where cluster characteristics can be visualized through traversals along cluster directions. Results on MNIST and FashionMNIST datasets demonstrate the model's ability to learn meaningful cluster representations.

Graph neural networks (GNNs) have substantially advanced machine learning applications to graph-structured data by effectively propagating node attributes end-to-end. Typically, GNNs rely on the assumption of homophily, where nodes with similar labels are more likely to be connected [39, 36]. The homophily assumption holds true in contexts such as social networks and citation graphs, where models like GCN [14], GIN [37], and GraphSAGE [11] excel at tasks like node classification and graph prediction. However, this is not the case in heterophilous datasets, such as web page and transaction networks, where edges often link nodes with differing labels. Models such as GAT [35] and various graph transformers [38, 9] show improved performance on these datasets. With their attention mechanisms that learns edge importances, they reduce the dependency on the homophily. In this setting, our work specifically addresses unsupervised attributed node clustering tasks, which require models to function without any label information during training.

PCA, but rather in another model based on similar In this paper, we characterize Probabilistic Principal principles. Component Analysis in Hilbert spaces and demonstrate how the optimal solution admits a More recently, Restricted Kernel Machines (Suykens, 2017) representation in dual space. This allows us to develop opened a new door for a probabilistic version of PCA both a generative framework for kernel methods. in primal and dual. They essentially use the Fenchel-Young Furthermore, we show how it englobes Kernel inequality on a variational formulation of KPCA (Suykens Principal Component Analysis and illustrate its et al., 2003; Alaíz et al., 2018) to obtain an energy function, working on a toy and a real dataset.

While kernel methods have shown excellent performance and generalization capabilities, they tend to fall behind when We propose a unifying setting that combines existing it comes to large-scale problems due to their memory and restricted kernel machine methods into a single computational complexity. Additionally, it can be difficult primal-dual multi-view framework for kernel to change their architecture to allow for hierarchical principal component analysis in both supervised representation learning, which is one of the most powerful and unsupervised settings. We derive the primal capabilities of neural networks. Recently, Restricted and dual representations of the framework and Kernel Machines (RKM), were proposed which connect relate different training and inference algorithms least-squares support vector machines and kernel principal from a theoretical perspective. We show how to component analysis (kernel PCA) with Restricted Boltzmann achieve full equivalence in primal and dual formulations machines (Suykens, 2017). RKMs extend the primal by rescaling primal variables. Finally, and dual model representations present in least-squares support we experimentally validate the equivalence and vector machines, from shallow to deep architectures by provide insight into the relationships between different introducing the dual variables as hidden features through methods on a number of time series data conjugate feature duality. This provides a framework of sets by recursively forecasting unseen test data kernel methods represented by visible and hidden units as and visualizing the learned features.

In this paper, we propose a kernel principal component analysis model for multi-variate time series forecasting, where the training and prediction schemes are derived from the multi-view formulation of Restricted Kernel Machines. The training problem is simply an eigenvalue decomposition of the summation of two kernel matrices corresponding to the views of the input and output data. When a linear kernel is used for the output view, it is shown that the forecasting equation takes the form of kernel ridge regression. When that kernel is non-linear, a pre-image problem has to be solved to forecast a point in the input space. We evaluate the model on several standard time series datasets, perform ablation studies, benchmark with closely related models and discuss its results.

Multi-view Spectral Clustering (MvSC) attracts increasing attention due to diverse data sources. However, most existing works are prohibited in out-of-sample predictions and overlook model interpretability and exploration of clustering results. In this paper, a new method for MvSC is proposed via a shared latent space from the Restricted Kernel Machine framework. Through the lens of conjugate feature duality, we cast the weighted kernel principal component analysis problem for MvSC and develop a modified weighted conjugate feature duality to formulate dual variables. In our method, the dual variables, playing the role of hidden features, are shared by all views to construct a common latent space, coupling the views by learning projections from view-specific spaces. Such single latent space promotes well-separated clusters and provides straightforward data exploration, facilitating visualization and interpretation. Our method requires only a single eigendecomposition, whose dimension is independent of the number of views. To boost higher-order correlations, tensor-based modelling is introduced without increasing computational complexity. Our method can be flexibly applied with out-of-sample extensions, enabling greatly improved efficiency for large-scale data with fixed-size kernel schemes. Numerical experiments verify that our method is effective regarding accuracy, efficiency, and interpretability, showing a sharp eigenvalue decay and distinct latent variable distributions.