Understanding LSTM Networks -- colah's blog

As you read this essay, you understand each word based on your understanding of previous words. You don't throw everything away and start thinking from scratch again. Traditional neural networks can't do this, and it seems like a major shortcoming. For example, imagine you want to classify what kind of event is happening at every point in a movie. It's unclear how a traditional neural network could use its reasoning about previous events in the film to inform later ones. Recurrent neural networks address this issue.