Singular Value Decomposition

Some quick notes about Singular Value Decomposition (SVD) to develop an intuition that will help solve problems related to collaborative filtering, natural language processing (NLP), dimensionality reduction, image compression, denoising data etc. Let's imagine we have a matrix \(\textbf{A}\) of size \(m\) by \(n\) where \(m\) is the number of rows and \(n\) is the number of columns. If we were building a recommender system, we can think of each row representing a user, each column representing an item, and each element in the matrix indicating whether the user has interacted with the item. In NLP, we can think of each row representing a document, and each column representing a term. This matrix stores the left singular vectors.