Decoders built on Gaussian processes (GPs) are enticing due to the marginalisation over the non-linear function space. Such models (also known as GP-LVMs) are often expensive and notoriously difficult to train in practice, but can be scaled using variational inference and inducing points. In this paper, we revisit active set approximations. We develop a new stochastic estimate of the log-marginal likelihood based on recently discovered links to cross-validation, and we propose a computationally efficient approximation thereof. We demonstrate that the resulting stochastic active sets (SAS) approximation significantly improves the robustness of GP decoder training, while reducing computational cost. The SAS-GP obtains more structure in the latent space, scales to many datapoints, and learns better representations than variational autoencoders, which is rarely the case for GP decoders.

We develop a module-based method that having a dictionary of well fitted GPs, one could build ensemble GP models without revisiting any data.

For example, census data are usually sampled or collected as aggregated at different administrative divisions, e.g.

Neural Information Processing Systems http://nips.cc/

Indoing so, we generalize existing approaches and offer information-theoretic corrections and efficient variational approximations.

While existing approaches only model finite, discrete fidelities, in practice, the feasible fidelity choice is often infinite, which can correspond to a continuous mesh spacing orfinite element length.

Variational inference was employed in prior work to inferz (and w implicitly), and to obtain a point estimate ofθ, as a by-product of optimising the variational lower bound. Critically, in this representation weights are only implicitly parametrized through the use of these latent variables, which transforms inference onweights into inference ofthemuch smaller collection oflatent unit variables.

The approximate posterior distributions of the latent variables are derived with variational inference, and the evidence lower bound is evaluated and optimized by the proposed recursive sampling scheme.

One of the principal assumptions of the GP-LVM is that the prior p(f) regularises the smoothness of all mappings equally, so we only consider onekernel.