The mathematics and Intuitions of Principal Component Analysis (PCA) Using Truncated Singular…

As data scientists or Machine learning experts, we are faced with tonnes of columns of data to extract insight from, among these features are redundant ones, in more fancier mathematical term -- co-linear features. The numerous columns of features without prior treatment leads to curse of dimensionality which in turn leads to over fitting. To ameliorate this curse of dimensionality, principal component analysis (PCA for short) which is one of many ways to address this, is employed using truncated Singular Value Decomposition (SVD). Principal Component Analysis starts to make sense when the number of measured variables are more than three (3) where visualization of the cloud of the data point is difficult and it is near impossible to get insight from. First: Let's try to grasp the goal of Principal Component Analysis.