Generative Principal Component Analysis

In this paper, we study the problem of principal component analysis with generative modeling assumptions, adopting a general model for the observed matrix that encompasses notable special cases, including spiked matrix recovery and phase retrieval. The key assumption is that the underlying signal lies near the range of an L-Lipschitz continuous generative model with bounded k-dimensional inputs. Moreover, we provide a variant of the classic power method, which projects the calculated data onto the range of the generative model during each iteration. We show that under suitable conditions, this method converges exponentially fast to a point achieving the above-mentioned statistical rate. We perform experiments on various image datasets for spiked matrix and phase retrieval models, and illustrate performance gains of our method to the classic power method and the truncated power method devised for sparse principal component analysis. Principal component analysis (PCA) is one of the most popular techniques for data processing and dimensionality reduction [1], with an abundance of applications such as image recognition [2], gene expression data analysis [3], and clustering [4], [5]. PCA seeks to find the directions that capture maximal variances in vector-valued data. Z. Liu is with the Department of Computer Science, National University of Singapore (email: dcslizha@nus.edu.sg).