A graphical model is a structured representation of the data generating process. The traditional method to reason over random variables is to perform inference in this graphical model. However, in many cases the generating process is only a poor approximation of the much more complex true data generating process, leading to suboptimal estimations.

We address the reviewer comments below. R1: More discussion about how this idea could be applied to other generative models. Belief Propagation messages, then the model would also be able to run on discrete variables. Currently, in the paper, we only show the plots for J=1 and J=2 expansion terms. Therefore, we don't see as a limitation the fact that the functional form is Given the functional form, our method has the advantage to combine it with deep learning. R2: Missing citation to related work: http://proceedings.mlr

A graphical model is a structured representation of the data generating process. The traditional method to reason over random variables is to perform inference in this graphical model. However, in many cases the generating process is only a poor approximation of the much more complex true data generating process, leading to suboptimal estimations.

We address the reviewer comments below. R1: More discussion about how this idea could be applied to other generative models. Belief Propagation messages, then the model would also be able to run on discrete variables. Currently, in the paper, we only show the plots for J=1 and J=2 expansion terms. Therefore, we don't see as a limitation the fact that the functional form is Given the functional form, our method has the advantage to combine it with deep learning. R2: Missing citation to related work: http://proceedings.mlr

Overall this is a nice idea that works on using black box models to amortize the residuals from doing inference assuming a linearized approximation to the model. I found the experiments to be well organized albeit mostly on small scale/synthetic data. Summary: This paper introduces a procedure for combining graph neural networks with traditional methods for probabilistic inference (instantiated in HMMs). When we have linear dynamics in a HMM, inference is exact. For nonlinear dynamics, when we have access to the functional form of the true dynamics of the state space model, we can linearize the transition and emission functions (via a Taylor expansion) and represent them as matrices.