Reviews: Cortical microcircuits as gated-recurrent neural networks

A new recurrent neural network model is presented. It has similar functional features as LSTMs but additive gating instead of multiplicative gating for the input and output gates. With this additive gating mechanism there are some striking similarities to cortical circuits. In my opinion, this papers could be really interesting for both computational neuroscientists and machine learners. For small network sizes the proposed model performs as good or better than simple LSTMs on some non-trivial tasks. It is, however, somewhat disappointing that For the language modelling task it seems, however, that the multiplicative forget gates are still needed for good performance in larger networks.