Adding A Custom Attention Layer To Recurrent Neural Network In Keras

Deep learning networks have gained immense popularity in the past few years. The'attention mechanism' is integrated with the deep learning networks to improve their performance. Adding attention component to the network has shown significant improvement in tasks such as machine translation, image recognition, text summarization and similar applications. This tutorial shows how to add a custom attention layer to a network built using a recurrent neural network. We'll illustrate an end to end application of time series forecasting using a very simple dataset.