Forecasting of the development of a partially-observed dynamical time series with the aid of time-invariance and linearity

Notwithstanding its difficulty, forecasting of the development of intricate non-linear dynamical systems has been in a spotlight of various scientific fields (Strogatz, 2001; Jackson and Radunskaya, 2015). A plausible approach to forecasting the development is to isolate the non-linear estimation problem into (i) learning non-linear representations by applying highly non-linear functions such as deep neural networks (Goodfellow et al., 2016), and (ii) estimating its development with simple linear models. An example is a reservoir computing (RC; Jaeger, 2001, 2002). RC first randomly specifies a state in the reservoir layer in recurrent neural network (Rumelhart et al., 1986), and optimizes the weights only in the output layer; RC corresponds to non-linearly transform its input (in the reservoir layer) and trains a simple linear prediction model (in the output layer). It has been reported that such a simple combination of the non-linear representation learning and the linear estimation is effective to forecasting the intricate dynamical systems (Tanaka et al., 2019). Effectiveness of the simple combination is not limited to RC; applying a linear model to the non-linear representation in more general deep neural network is also regarded as a solid forecasting strategy (Lusch et al., 2018). Unfortunately, however, partial degrees of freedom corresponding to several state variables are not observed in some practical situations (Lucor et al., 2022; Cheng et al., 2023). There could be a variety of reasons for missing observations: it would be caused by the difficulty of measurement, it would be caused by the immature understanding of the system of interest, and so forth.