Appendix A Extended Background 18 A.1 Two Different RNNs 18 A.2 Contraction Math 18 A.2.1 Feedback and Hierarchical Combinations

A.1 Two Different RNNs Note that in neuroscience, the variable x in equation (1) is typically thought of as a vector of neural membrane potentials. It was shown in [Miller and Fumarola, 2012] that the RNN (1) is equivalent via an affine transformation to another commonly used RNN model, τẏ = y + φ(Wy + b(t)) (4) where the variable y is interpreted as a vector of firing rates, rather than membrane potentials.