[P] A PyTorch module implementing an Echo State Network • r/MachineLearning

Recurrent neural networks when activated can remain active for a time following that activation as neurons within the network continue to excite other neurons, even in the absence of external inputs. These internal activations contain some information about the inputs into the network. The "echo state" refers to these internal reverberations which give the RNN an implicit memory, even when the neurons themselves are stateless (i.e. Even randomly connected and weighted RNNs have these properties. As it concerns machine learning, the important property of the RNN is how long the information can be stored in these "echos".