Modern clustering approaches often trade interpretability for performance, particularly in deep learning-based methods. We present Generative Kernel Spectral Clustering (GenKSC), a novel model combining kernel spectral clustering with generative modeling to produce both well-defined clusters and interpretable representations. By augmenting weighted variance maximization with reconstruction and clustering losses, our model creates an explorable latent space where cluster characteristics can be visualized through traversals along cluster directions. Results on MNIST and FashionMNIST datasets demonstrate the model's ability to learn meaningful cluster representations.

Graph neural networks (GNNs) have substantially advanced machine learning applications to graph-structured data by effectively propagating node attributes end-to-end. Typically, GNNs rely on the assumption of homophily, where nodes with similar labels are more likely to be connected [39, 36]. The homophily assumption holds true in contexts such as social networks and citation graphs, where models like GCN [14], GIN [37], and GraphSAGE [11] excel at tasks like node classification and graph prediction. However, this is not the case in heterophilous datasets, such as web page and transaction networks, where edges often link nodes with differing labels. Models such as GAT [35] and various graph transformers [38, 9] show improved performance on these datasets. With their attention mechanisms that learns edge importances, they reduce the dependency on the homophily. In this setting, our work specifically addresses unsupervised attributed node clustering tasks, which require models to function without any label information during training.

We present a deep Graph Convolutional Kernel Machine (GCKM) for semi-supervised node classification in graphs. The method is built of two main types of blocks: (i) We introduce unsupervised kernel machine layers propagating the node features in a one-hop neighborhood, using implicit node feature mappings. (ii) We specify a semi-supervised classification kernel machine through the lens of the Fenchel-Young inequality. We derive an effective initialization scheme and efficient end-to-end training algorithm in the dual variables for the full architecture. The main idea underlying GCKM is that, because of the unsupervised core, the final model can achieve higher performance in semi-supervised node classification when few labels are available for training. Experimental results demonstrate the effectiveness of the proposed framework.

While kernel methods have shown excellent performance and generalization capabilities, they tend to fall behind when We propose a unifying setting that combines existing it comes to large-scale problems due to their memory and restricted kernel machine methods into a single computational complexity. Additionally, it can be difficult primal-dual multi-view framework for kernel to change their architecture to allow for hierarchical principal component analysis in both supervised representation learning, which is one of the most powerful and unsupervised settings. We derive the primal capabilities of neural networks. Recently, Restricted and dual representations of the framework and Kernel Machines (RKM), were proposed which connect relate different training and inference algorithms least-squares support vector machines and kernel principal from a theoretical perspective. We show how to component analysis (kernel PCA) with Restricted Boltzmann achieve full equivalence in primal and dual formulations machines (Suykens, 2017). RKMs extend the primal by rescaling primal variables. Finally, and dual model representations present in least-squares support we experimentally validate the equivalence and vector machines, from shallow to deep architectures by provide insight into the relationships between different introducing the dual variables as hidden features through methods on a number of time series data conjugate feature duality. This provides a framework of sets by recursively forecasting unseen test data kernel methods represented by visible and hidden units as and visualizing the learned features.