Creating A LSTM Network From Scratch

LSTMs are a type of recurrent neural network that works better than simple recurrent neural networks as they learn to prioritise and ignore data, adding a further layer of complexity to the neural network, therefore allowing the neural network to learn more complex patterns. The LSTM architecture does not consist of just one neural network but a battery of at least three neural networks that are trained simultaneously. Additionally, LSTM architecture also contains gates that give certain pieces of data a higher weightage to the ultimate prediction of the neural network. In Figure 1.1 the line at the right side of the chart is the cell state of the neural network. One can think of this as the assembly line of the LSTM: Each of the separate neural networks feed into the cell state.