We thank all reviewers for their valuable time and feedback. Note that multiple recent works (offline and online) simply assume a linear MDP with known features in analysis. KL-divergence formulation to impose different distribution priors when available. We agree about Section 3.3 and in retrospect should have saved We will remove it and move some of the Appendix into the paper. We will add references to maximum-entropy approaches in RL and IRL.

We demonstrate that BootDQNprior+'s lagged target parameters, which are essential to its performance, arise from applying approximate inference to the BBO posterior.

Below, we proceed to describe our contributions in greater detail. Our framework builds upon the seminal work of Jordan et al. [1998], which introduced the celebrated

Gaussian Process models are a rich distribution over functions with inductive biases controlled by a kernel function. Learning occurs through optimisation of the kernel hyperparameters using the marginal likelihood as the objective.

One of the fundamental aspects of over-parametrized models is that they are capable of interpolating the training data.