The Essence of Principal Component Analysis (PCA)

This is one of the most fascinating ideas in Linear Algebra. By multiplying a matrix to a vector, we linearly transform that vector. If you feel like your grip on basic linear algebra is a little loose, I strongly recommend that you watch 3b1b's series on Linear Algebra. A non-trivial vector whose span doesn't change upon being multiplied by a matrix is an eigenvector of that matrix. Now, let me clarify two things here- firstly, the span loosely means the direction of that vector and secondly, although the direction doesn't change, the magnitude can. How stretched or squished the eigenvector becomes i.e. the factor by which the magnitude changes upon multiplication is called the eigenvalue of that eigenvector.