Reviews: Input-Cell Attention Reduces Vanishing Saliency of Recurrent Neural Networks

The paper starts by showing empirically and theoretically that saliency maps generated using gradient vanishes over long sequences in LSTMs. The authors propose a modification to the LSTM cell, called LSTM with cell-attention, which can attend to all previous time steps. They show that this approach improve considerably the saliency on the input sequence. They also test their approach on the fMRI dataset of the Human Connectome Project (HCP). Originality: The proposed LSTM with cell-attention is a novel combination of well-known techniques.