Exploring LSTMs

The first time I learned about LSTMs, my eyes glazed over. It turns out LSTMs are a fairly simple extension to neural networks, and they're behind a lot of the amazing achievements deep learning has made in the past few years. So I'll try to present them as intuitively as possible – in such a way that you could have discovered them yourself. Imagine we have a sequence of images from a movie, and we want to label each image with an activity (is this a fight?, are the characters talking?, are the characters eating?). One way is to ignore the sequential nature of the images, and build a per-image classifier that considers each image in isolation.