Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systems

Neural Information Processing Systems 

They learn to transform input sequences into output sequences by using an internal state that allows data from the past to influence the current state.