Understanding Recurrent Neural Networks

The first time I came across RNNs, I was completely baffled. How can a network even remember things? Recurrent Neural Networks have proved to be effective and popular for processing sequential data ever since the first time they emerged in the late 1980s. Recurrent Neural Networks have been derived from vanilla Feed Forward Neural Networks. They have so-called memory elements that help the network remember previous outputs. They are Recurrent because they repeatedly perform the same task for every element in the sequence, with the output being dependent on the previous computations. Recurrent Neural Networks (RNNs) have been a huge improvement over the vanilla neural network. A typical vanilla neural network calculates an output on the current input and weights with a limitation of predetermined fixed input size. In this article, we will go over the architecture of RNNs, with just enough math by taking the example of Elman Network.