Non-Negative Matrix Factorization with Scale Data Structure Preservation

Low-rank matrix factorization (MF) is a hot topic in many research problems such as feature extraction and dimensionality reduction Vidal et al. [2005], subspace segmentation Liu et al. [2010], data clustering Favaro et al. [2011], image processing and computer vision Peng et al. [2012] to mention a few. The key idea behind MF is that there is a latent data structure embedded in the high dimensional observed data which, once discovered, provides better capacity for learning. Formally, MF techniques aim to decompose an observed high-dimensional data matrix into its constitute lower-dimensional factorizing matrices (in general two). One of the factorizing matrices represents the lower-dimensional space and the other one represents the spread of latent data in that space. MF has been widely used as a unified technique for dimensionality reduction, clustering, and matrix completion. There are several variants of MF in the literature including basic MF (BMF), non-negative MF (NMF) and Orthogonal NMF (ONMF). BMF are those described using traditional matrix decomposition such as principal component analysis (PCA), vector quantization (VQ) and singular value decomposition (SVD).