Spectral embedding of regularized block models

A BSTRACT Spectral embedding is a popular technique for the representation of graph data. Several regularization techniques have been proposed to improve the quality of the embedding with respect to downstream tasks like clustering. In this paper, we explain on a simple block model the impact of the complete graph regularization, whereby a constant is added to all entries of the adjacency matrix. Specifically, we show that the regularization forces the spectral embedding to focus on the largest blocks, making the representation less sensitive to noise or outliers. We illustrate these results on both on both synthetic and real data, showing how regularization improves standard clustering scores. 1 I NTRODUCTION Spectral embedding is a standard technique for the representation of graph data (Ng et al., 2002; Belkin & Niyogi, 2002). Given the adjacency matrix A R n n of the graph, it is obtained by solving either the eigenvalue problem: LX X Λ, with X T X I, (1) or the generalized eigenvalue problem: LX DX Λ, with X T DX I, (2) where D diag( A 1 n) is the degree matrix, with 1 n the all-ones vector of dimension n, L D A is the Laplacian matrix of the graph, Λ R k k is the diagonal matrix of the k smallest (generalized) eigenvalues of L and X R n k is the corresponding matrix of (generalized) eigenvectors. In this paper, we only consider the generalized eigenvalue problem, whose solution is given by the spectral decomposition of the normalized Laplacian matrix L norm I D 1/ 2 AD 1/ 2 (Luxburg, 2007). The spectral embedding can be interpreted as equilibrium states of some physical systems (Snell & Doyle, 2000; Spielman, 2007; Bonald et al., 2018), a desirable property in modern machine learning.