The Gaussian process state space model (GPSSM) is a non-linear dynamical system, where unknown transition and/or measurement mappings are described by GPs. Most research in GPSSMs has focussed on the state estimation problem, i.e., computing a posterior of the latent state given the model. However, the key challenge in GPSSMs has not been satisfactorily addressed yet: system identification, i.e., learning the model. To address this challenge, we impose a structured Gaussian variational posterior distribution over the latent states, which is parameterised by a recognition model in the form of a bi-directional recurrent neural network. Inference with this structure allows us to recover a posterior smoothed over sequences of data. We provide a practical algorithm for efficiently computing a lower bound on the marginal likelihood using the reparameterisation trick. This further allows for the use of arbitrary kernels within the GPSSM. We demonstrate that the learnt GPSSM can efficiently generate plausible future trajectories of the identified system after only observing a small number of episodes from the true system.

We thank the reviewers for their close reading of the paper and helpful feedback. Forexample, one can use thedensity ratio estimates7 provided by DualDICE to modify (importance-weight) the off-policy data distribution before passing it to a policy8 gradient orQ-learning method. The figures are overall too small... In Figure 2 the x axis label is missing. The x-axis is training step.

See Appendix Aforanoteontrain/testsplitfor Task 3. loss Testloss Energy Baseline HNNBaseline HNNBaseline HNN mass-spring170 20.38 .1 pendulum 42 10 25 5 pendulum 390 7 14 5 (6.3e4 3e4 39 5 pendulum.3

Neural Information Processing Systems http://nips.cc/

However,inpractice, we did not find both effects inthe training. The last term in the loss function of COMET does not need the information from the dataset. This section contains the experiment details for cases tested in section 4. In section 4, there are 6 simple experiments performed todemonstrate the capability ofCOMET:(1) mass-spring, (2) 2D Case5: 2Dnonlinear spring.Weconsider acaseofamotion ofanobjectofmassm=1in2D where it is connected to the origin with a nonlinear spring with forceF = |r|2r where r is the position of the object in 2D coordinate. The constants of motion of this systems are the energy and the angular momentum,whichmakesnc=2. Case 6: Lotka-Volterraequation isanordinary differential equation modelling thepopulation of predatorandprey. The training loss in this case was composed of the reconstruction loss and the dynamics loss.

Useful biases often exploit symmetries intheprediction problem, suchasconvolutional networks relying on translation equivariance.