Long-time predictive modeling of nonlinear dynamical systems using neural networks

Complexity manuscript No. (will be inserted by the editor) Abstract We study the use of feedforward neural networks (FNN) to develop models of nonlinear dynamical systems from data. Emphasis is placed on predictions at long times, with limited data availability. Inspired by global stability analysis, and the observation of strong correlation between the local error and the maximum singular value of the Jacobian of the ANN, we introduce Jacobian regularization in the loss function. This regularization suppresses the sensitivity of the prediction to the local error and is shown to improve accuracy and robustness. Comparison between the proposed approach and sparse polynomial regression is presented in numerical examples ranging from simple ODE systems to nonlinear PDE systems including vortex shedding behind a cylinder, and instability-driven buoyant mixing flow. Furthermore, limitations of feedforward neural networks are highlighted, especially when the training data does not include a low dimensional attractor. Strategies of data augmentation are presented as remedies to address these issues to a certain extent. Keywords data-driven modeling · artificial neural networks · Jacobian regularization · machine learning 1 Introduction The need to model dynamical behavior from data is pervasive across science and engineering. Applications are found in diverse domains such as in control systems [44], time series modeling [40], and describing the evolution of coherent structures [6]. While data-driven modeling of dynamical systems can be broadly classified as a special case of system identification [25], it is important to note certain distinguishing qualities: the learning process may be performed off-line; physical systems may involve very high dimensions; and the goal may involve the prediction of longtime behaviour from limited training data.