Hierarchical Temporal Representation in Linear Reservoir Computing

In the last years, the extension of deep neural network architectures towards recurrent processing of temporal data has opened the way to novel approaches to effectively learn hierarchical representations of time-series featured by multiple timescales dynamics [19, 18, 10, 9, 1]. Recently, within the umbrella of randomized neural network approaches [4], Reservoir Computing (RC) [21, 15] has proved to be a useful tool for analyzing the intrinsic properties of stacked architectures in recurrent neural networks (RNNs), allowing at the same time to exploit the extreme efficiency of RC training algorithms in the design of novel deep RNN models. Stemming from the Echo State Network (ESN) approach [12] the study of the dynamics of multi-layered recurrent reservoir architectures has been introduced with the deep-ESN model in [7, 5].