Sparse outlier-robust PCA for multi-source data

Principal component analysis (PCA) is undoubtedly one of the most important unsupervised statistical methods available. The basic idea is to project the observations in a given dataset onto a new vector space with orthonormal basis where each basis vector is a linear combination of the original variables constructed to capture the highest variability for the first basis vector, the second highest variability for the second basis vector and so on. The new variables are called Principal Components (PC), the coordinates of the PCs in the original variable space are called loadings and the coordinates of the observations with respect to the PCs are called scores. Often, only the first few PCs that catch a majority of the variance and thus of the available information are analyzed. As such, PCA finds widespread application across numerous areas, such as dimensionality reduction, visualization, clustering, feature engineering and many more. For standard PCA we get loadings that are often a combination of all variables involved. Especially nowadays with datasets consisting of many variables, sensible, efficient and correct interpretation of scores and loadings can get difficult. Moreover, by implicitly (or also explicitly) focusing the interpretation on large (absolute) loading entries and ignoring small ones, misleading interpretation results can be produced as discussed in Cadima and Jolliffe (1995). Therefore, induced sparsity in the loading entries is necessary to ensure correct interpretation of PCA results.