Recurrent neural networks and LSTM tutorial in Python and TensorFlow - Adventures in Machine Learning

In the deep learning journey so far on this website, I've introduced dense neural networks and convolutional neural networks (CNNs) which explain how to perform classification tasks on static images. We've seen good results, especially with CNN's. However, what happens if we want to analyze dynamic data? There are ways to do some of this using CNN's, but the most popular method of performing classification and other analysis on sequences of data is recurrent neural networks. This tutorial will be a very comprehensive introduction to recurrent neural networks and a subset of such networks – long-short term memory networks (or LSTM networks). I'll also show you how to implement such networks in TensorFlow – including the data preparation step. It's going to be a long one, so settle in and enjoy these pivotal networks in deep learning – at the end of this post, you'll have a very solid understanding of recurrent neural networks and LSTMs. As always, all the code for this post can be found on this site's Github repository. Recommended online course: If you are more of a video course learner, I'd recommend this inexpensive Udemy course: Deep Learning: Recurrent Neural Networks in Python A recurrent neural network, at its most fundamental level, is simply a type of densely connected neural network (for an introduction to such networks, see my tutorial). However, the key difference to normal feed forward networks is the introduction of time – in particular, the output of the hidden layer in a recurrent neural network is fed back into itself. In the diagram above, we have a simple recurrent neural network with three input nodes. These input nodes are fed into a hidden layer, with sigmoid activations, as per any normal densely connected neural network. What happens next is what is interesting – the output of the hidden layer is then fed back into the same hidden layer.