Understanding the link between PCA and Eigenvectors

In my former article, I've been introducing the concept of Principal Components Analysis, a powerful technique used in Machine Learning to reduce the dimensionality of your input data. An important mathematical feature of PCA is its link with the Eigenvectors (and Eigenvalues) of the original features matrix X, and in this article, I'm going to show the ratio behind this link. So let's start with a brief recap of PCA. Differently from features selection techniques, PCA does not use a subset of the initial features: it rather computes a new set of features obtained as linear combinations of the original ones, living in a lower-dimensional space. Where M is at most as big as p, but of course, because of the purpose of the task (dimensionality reduction), it is chosen to be less than p. The coefficients of the first 2 lines are known as loading factors and are computed in such a way that these new variables are uncorrelated and most of the information within the initial variables is stored into the first components.