Fast Riemannian-manifold Hamiltonian Monte Carlo for hierarchical Gaussian-process models

Hierarchical Bayesian models based on Gaussian processes a re considered useful for describing complex nonlinear statistical dependen cies among variables in real-world data. However, effective Monte Carlo algorithm s for inference with these models have not yet been established, except for sever al simple cases. In this study, we show that, compared with the slow inference ac hieved with existing program libraries, the performance of Riemannian-m anifold Hamiltonian Monte Carlo (RMHMC) can be drastically improved by optimisi ng the computation order according to the model structure and dynamical ly programming the eigendecomposition. This improvement cannot be achieved w hen using an existing library based on a naive automatic differentiator. W e nu merically demonstrate that RMHMC effectively samples from the posterior, allowin g the calculation of model evidence, in a Bayesian logistic regression on simula ted data and in the estimation of propensity functions for the American nation al medical expenditure data using several Bayesian multiple-kernel models. These results lay a foundation for implementing effective Monte Carlo algorithms for analysing real-world data with Gaussian processes, and highlight the need to deve lop a customisable library set that allows users to incorporate dynamically pr ogrammed objects and finely optimises the mode of automatic differentiation depe nding on the model structure.