Iteratively Reweighted Least Squares Algorithms for L1-Norm Principal Component Analysis

Principal component analysis (PCA) is a technique to find orthonormal vectors, which are a linear combination of the attributes of the data, that explain the variance structure of the data [12]. Since a few orthonormal vectors usually explain most of the variance, PCA is often used to reduce dimension of the data by keeping only a few of the orthonormal vectors. These orthonormal vectors are called principal components (PCs). For dimensionality reduction, we are given target dimension p, the number of PCs. To measure accuracy, given p principal components, first, the original data is projected into the lower dimension using the PCs. Next, the projected data in the lower dimension is lifted to the original dimension using the PCs. Observe that this procedure causes loss of some information if p is smaller than the dimension of the original attribute space. The reconstruction error is defined by the difference between the projected-and-lifted data and the original data. To select the best p PCs, the following two objective functions are usually used: [P1] minimization of the reconstruction error, [P2] maximization of the variance of the projected data.