Constraining Chaos: Enforcing dynamical invariants in the training of recurrent neural networks

Predicting the future trajectory of a dynamical system--a time series whose evolution is governed by a set of differential equations--is crucial in fields such as weather prediction, economics, chemistry, physics and many others [1, 2]. A prediction can be generated by deriving the governing equations of motion (EOM) for the system and integrating forward in time, perhaps with data being used to determine the value of particular constants or the initial conditions. Machine learning (ML), on the other hand, allows the construction of a forecast purely from observational data in lieu of a physical model. When the EOM are expensive to evaluate numerically, ML can be used to construct a surrogate model; such models can be integrated into data assimilation [3] algorithms--such as the Kalman filter [4, 5]--a typical use case when data are noisy and the model imperfect, such as in numerical weather prediction [6]. The inclusion of physical knowledge--EOM, conservation laws and dynamical invariants--into ML algorithms has been a topic of ongoing interest [7-15].