Efficient identification of linear, parameter-varying, and nonlinear systems with noise models

However, if one goes beyond the well-established linear time-invariant (L TI) system identification framework [17], considering, e.g., linear time-varying (L TV) [22], linear parameter-varying (LPV) [29], or various nonlinear (NL) models [11], a core question of the resulting identification problems is how to efficiently estimate from data the functional relations involved in these models. While classical beyond-LTI methods used fixed basis-function parameteriza-tions to express nonlinearities, with the recent progress of computational capabilities and machine learning/deep learning methods, a wide-range of learning approaches has been introduced, from Gaussian process (GP) regression [24] and support vector machines [27] to artificial neural networks (ANN) based training (see [23] for a recent overview), including state-space (SS) recurrent networks such as LSTMs [13, 19], SS-ANNs [1-3, 10, 20, 25, 28], etc. In particular, the latter class of methods successfully combines deep learning with state-space model identification concepts, achieving extreme accuracy levels on well established benchmarks, see [1]. However, in the quest of solving the underlying function estimation problem with improved accuracy and reliability, many aspects of the classical identification theory have been sacrificed. While deep-learning methods can achieve remarkable results, the relating training time when no state-measurements are available is often too high compared to classical identification methods, requiring tens of hours, sometimes days, with the most sophisticated stochastic gradient methods, like Adam [15], to train mid-sized models on training sequences of reasonable length. Using such methods in an identification cycle, which requires iterations on model structure selection, hyperparameter optimization on validation data, and experiment design, is simply impractical. Next to the computational challenges, many of the developed machine learning-based identification methods do not consider measurement or process noise affecting the estimation problem, or are based on simplistic noise settings / isolated scenarios, with a few exceptions, e.g., [1, 25]. The general understanding of process and noise models, and the related concept of one/ n -step-ahead predictors and connected modelling choices, model structures, and the overall estimation concept, which are well-developed for the L TI case, is largely missing in the general NL context. Besides theoretical limitations, in practice it is not clear to the user how to co-estimate linear and nonlinear elements, select a combination of process and noise models, and scale up the identification process from linear to nonlinear modeling within the same framework.