In machine learning, we need lots of data to build an efficient model, but dealing with a larger dataset is not an easy task we need to work hard in preprocessing the data and as a data scientist we will come across a situation dealing with a large number of variables here PCA (principal component analysis) is dimension reduction technique helps in dealing with those problems. In this article, we will demonstrate how to work on larger data and images using a famous dimension reduction technique PCA( principal component analysis). PCA is a dimensionality reduction that is often used to reduce the dimension of the variables of a larger dataset that is compressed to the smaller one which contains most of the information to build an efficient model. In a real-time scenario when you are working reducing the number of variables in the dataset you need compromise on model accuracy but using PCA will give good accuracy. The idea of PCA is to reduce the variables in the dataset and preserve data as much as possible.

This is one of the most fascinating ideas in Linear Algebra. By multiplying a matrix to a vector, we linearly transform that vector. If you feel like your grip on basic linear algebra is a little loose, I strongly recommend that you watch 3b1b's series on Linear Algebra. A non-trivial vector whose span doesn't change upon being multiplied by a matrix is an eigenvector of that matrix. Now, let me clarify two things here- firstly, the span loosely means the direction of that vector and secondly, although the direction doesn't change, the magnitude can. How stretched or squished the eigenvector becomes i.e. the factor by which the magnitude changes upon multiplication is called the eigenvalue of that eigenvector.

PCA provides valuable insights that reach beyond descriptive statistics and help to discover underlying patterns. Two PCA metrics indicate 1. how many components capture the largest share of variance (explained variance), and 2., which features correlate with the most important components (factor loading). These metrics crosscheck previous steps in the project work flow, such as data collection which then can be adjusted. When a project structure resembles the one below, the prepared dataset is under scrutiny in the 4. step by looking at descriptive statistics. Among the most common ones are means, distributions and correlations taken across all observations or subgroups.

Principal component analysis (PCA; Jolliffe (2002)) is one of the most popular methods used to find a low-dimensional subspace in which a given dataset lies. Classical PCA (cPCA) can be formulated as a problem to find a subspace that minimizes the sum of squared residuals, but squared residuals make PCA vulnerable to outliers. A lot of PCA algorithms have been proposed to robustify cPCA. The R1-PCA proposed by Ding et al. (2006) replaced the sum of squared residuals in cPCA with the sum of unsquared ones. The optimal solution of R1-PCA has similar properties to those of cPCA, that is, it is given as the eigenvectors of the weighted covariance matrix and it is rotationally invariant. The absolute residuals can reduce negative impact of outliers, but an arbitrary large outlier can still break down the estimate. More recently, Zhang and Lerman (2014) and Lerman et al. (2015) relaxed the optimization problem so that the set of projection matrices is extended to a set of convex set of matrices, and 1

Tensor, also known as multi-dimensional array, arises from many applications in signal processing, manufacturing processes, healthcare, among others. As one of the most popular methods in tensor literature, Robust tensor principal component analysis (RTPCA) is a very effective tool to extract the low rank and sparse components in tensors. In this paper, a new method to analyze RTPCA is proposed based on the recently developed tensor-tensor product and tensor singular value decomposition (t-SVD). Specifically, it aims to solve a convex optimization problem whose objective function is a weighted combination of the tensor nuclear norm and the l1-norm. In most of literature of RTPCA, the exact recovery is built on the tensor incoherence conditions and the assumption of a uniform model on the sparse support. Unlike this conventional way, in this paper, without any assumption of randomness, the exact recovery can be achieved in a completely deterministic fashion by characterizing the tensor rank-sparsity incoherence, which is an uncertainty principle between the low-rank tensor spaces and the pattern of sparse tensor.

As data scientists or Machine learning experts, we are faced with tonnes of columns of data to extract insight from, among these features are redundant ones, in more fancier mathematical term -- co-linear features. The numerous columns of features without prior treatment leads to curse of dimensionality which in turn leads to over fitting. To ameliorate this curse of dimensionality, principal component analysis (PCA for short) which is one of many ways to address this, is employed using truncated Singular Value Decomposition (SVD). Principal Component Analysis starts to make sense when the number of measured variables are more than three (3) where visualization of the cloud of the data point is difficult and it is near impossible to get insight from. First: Let's try to grasp the goal of Principal Component Analysis.

Data visualization has always been an essential part of any machine learning operation. It helps to get a very clear intuition about the distribution of data, which in turn helps us to decide which model is best for the problem, we are dealing with. Currently, with the advancement of machine learning, we more often need to deal with large datasets. The datasets are having a large number of features, and can only be visualized using a large feature space. Now, we can only visualize 2-dimensional planes but, visualization of data is also seems pretty necessary, as we saw in our discussion above. This is where Principal Component Analysis comes in.

Fully symmetric learning rules for principal component analysis can be derived from a novel objective function suggested in our previous work. We observed that these learning rules suffer from slow convergence for covariance matrices where some principal eigenvalues are close to each other. Here we describe a modified objective function with an additional term which mitigates this convergence problem. We show that the learning rule derived from the modified objective function inherits all fixed points from the original learning rule (but may introduce additional ones). Also the stability of the inherited fixed points remains unchanged. Only the steepness of the objective function is increased in some directions. Simulations confirm that the convergence speed can be noticeably improved, depending on the weight factor of the additional term.

We develop two methods for the following fundamental statistical task: given an $\epsilon$-corrupted set of $n$ samples from a $d$-dimensional sub-Gaussian distribution, return an approximate top eigenvector of the covariance matrix. Our first robust PCA algorithm runs in polynomial time, returns a $1 - O(\epsilon\log\epsilon^{-1})$-approximate top eigenvector, and is based on a simple iterative filtering approach. Our second, which attains a slightly worse approximation factor, runs in nearly-linear time and sample complexity under a mild spectral gap assumption. These are the first polynomial-time algorithms yielding non-trivial information about the covariance of a corrupted sub-Gaussian distribution without requiring additional algebraic structure of moments. As a key technical tool, we develop the first width-independent solvers for Schatten-$p$ norm packing semidefinite programs, giving a $(1 + \epsilon)$-approximate solution in $O(p\log(\tfrac{nd}{\epsilon})\epsilon^{-1})$ input-sparsity time iterations (where $n$, $d$ are problem dimensions).

In dimension reduction problems, the adopted technique may produce disparities between the representation errors of two or more different groups. For instance, in the projected space, a specific class can be better represented in comparison with the other ones. Depending on the situation, this unfair result may introduce ethical concerns. In this context, this paper investigates how a fairness measure can be considered when performing dimension reduction through principal component analysis. Since both reconstruction error and fairness measure must be taken into account, we propose a multi-objective-based approach to tackle the Fair Principal Component Analysis problem. The experiments attest that a fairer result can be achieved with a very small loss in the reconstruction error.