Lenny #2: Autoencoders and Word Embeddings -- A year of Artificial Intelligence.

I want to take a quick break from my reinforcement learning endeavor (more on that soon) to talk about an interesting unsupervised learning model: autoencoders. The basic idea behind autoencoders is dimensionality reduction -- I have some high-dimensional representation of data, and I simply want to represent the same data with fewer numbers. Let's take the example of a picture of a face. If you look at each individual pixel, you'll quickly realize that neighboring pixels are usually highly correlated (they have a very similar color). What if we could take out the redundancy and express that same image in a fraction of the numbers?