Principal Component Analysis (PCA) is a technique used to find the core components that underlie different variables. It comes in very useful whenever doubts arise about the true origin of three or more variables. There are two main methods for performing a PCA: naive or less naive. In the naive method, you first check some conditions in your data which will determine the essentials of the analysis. In the less-naive method, you set the those yourself, based on whatever prior information or purposes you had.

Below is an example combining PCA plots with code similar to the above. The properties in both languages spread the'a', 'h', and'v' categories much more clearly than the concepts do (see more). An example of this code is use is available here (with data here).

In case you are here the first time, you may want to go through my previous deep dives into principal component analysis. Take a look at my tutorial I and tutorial II. To recap, Principal Component Analysis is a way to reduce the dimensions in our data set. This should make our computations faster and help us make better predictions as well. Now that you a fair idea on how PCA works and want to implement this in your production models, you may want to see how to implement this.

Normally when we are applying any of the machine learning concepts, we need to deal with a lot of matrices. Each matrix may have a lot of features or dimensions and then we will need to do a lot of computation. It may be prohibitive to run all the computations in a production environment, not counting the added problem of overfitting. In many occasions, it is also very useful to visualize the data. Due to our limitations as human beings, we are not able to visualize higher dimensions.

In the previous post, we saw why we should be interested in Principal Component Analysis. In this post, we will do some deep dive and get to know how this is implemented. Now that you have some idea about how to change higher dimensions to lower dimensions, we will go through the below description which is shown in a jupyter notebook. I have downloaded the data of three companies that are in the Indian stock market from Quandl. We will try to understand the Indian ecosystem using this.

This paper presents a remarkably simple, yet powerful, algorithm termed Coherence Pursuit (CoP) to robust Principal Component Analysis (PCA). As inliers lie in a low dimensional subspace and are mostly correlated, an inlier is likely to have strong mutual coherence with a large number of data points. By contrast, outliers either do not admit low dimensional structures or form small clusters. In either case, an outlier is unlikely to bear strong resemblance to a large number of data points. Given that, CoP sets an outlier apart from an inlier by comparing their coherence with the rest of the data points. The mutual coherences are computed by forming the Gram matrix of the normalized data points. Subsequently, the sought subspace is recovered from the span of the subset of the data points that exhibit strong coherence with the rest of the data. As CoP only involves one simple matrix multiplication, it is significantly faster than the state-of-the-art robust PCA algorithms. We derive analytical performance guarantees for CoP under different models for the distributions of inliers and outliers in both noise-free and noisy settings. CoP is the first robust PCA algorithm that is simultaneously non-iterative, provably robust to both unstructured and structured outliers, and can tolerate a large number of unstructured outliers.

Principal components analysis (PCA) is a statistical technique that allows to identify underlying linear patterns in a data set so it can be expressed in terms of other data set of significatively lower dimension without much loss of information. The final data set should be able to explain most of the variance of the original data set by making a variable reduction. The final variables will be named as principal components. The following image depicts the activity diagram that shows each step of the principal components analysis that will be explained in detail later. In order to illustrate the process described in the previous diagram, we are going to make use of the following data set which has two dimensions.

Last week I looked at Singular Value Decomposition unsupervised machine learning technique as part of a four-part series on data science concepts for beginners. Remember that unsupervised machine learning is data driven rather than task driven (supervised machine learning). Today we'll be staying in the dimension reduction part of unsupervised machine learning as shown in the Cheat-sheet below and will talk about principal component analysis or PCA. In a similar manner to SVD, PCA is trying to reduce the number of dimensions for data exploration. The PCA method is trying to maximize variance of the data to make a predictive model and converts a set of possibly correlated variables into a set of linearly uncorrelated variables.

What if our data have way more than 3-dimensions? In the table is the average consumption of 17 types of food in grams per person per week for every country in the UK. The table shows some interesting variations across different food types, but overall differences aren't so notable. Let's see if PCA can eliminate dimensions to emphasize how countries differ. Already we can see something is different about Northern Ireland.

We consider an online version of the robust Principle Component Analysis (PCA), which arises naturally in time-varying source separations such as video foreground-background separation. This paper proposes a compressive online robust PCA with prior information for recursively separating a sequences of frames into sparse and low-rank components from a small set of measurements. In contrast to conventional batch-based PCA, which processes all the frames directly, the proposed method processes measurements taken from each frame. Moreover, this method can efficiently incorporate multiple prior information, namely previous reconstructed frames, to improve the separation and thereafter, update the prior information for the next frame. We utilize multiple prior information by solving $n\text{-}\ell_{1}$ minimization for incorporating the previous sparse components and using incremental singular value decomposition ($\mathrm{SVD}$) for exploiting the previous low-rank components. We also establish theoretical bounds on the number of measurements required to guarantee successful separation under assumptions of static or slowly-changing low-rank components. Using numerical experiments, we evaluate our bounds and the performance of the proposed algorithm. In addition, we apply the proposed algorithm to online video foreground and background separation from compressive measurements. Experimental results show that the proposed method outperforms the existing methods.