Under The Hood of Neural Networks. Part 2: Recurrent.

In Part 1 of this series, we have studied the Forward and Backward passes of a Feed Forward Fully-Connected network. In spite of the fact, that Feed Forward networks are widespread and find a lot of real-world applications, they have a main limitation. Feed Forward networks cannot handle sequential data. This means that they cannot work with inputs of different sizes and they do not store information about previous states (memory). Thus, in this article, we will talk about Recurrent Neural Networks (RNNs) allowing overcome named limitations.