Optimality in Noisy Importance Sampling

A wide range of modern applications, especially in Bayesian inference framework [1], require the study of probability density functions (pdfs) which can be evaluated stochastically, i.e., only noisy evaluations can be obtained [2, 3, 4, 5]. For instance, this is the case of the pseudo-marginal approaches and doubly intractable posteriors [6, 7], approximate Bayesian computation (ABC) and likelihood-free schemes [8, 9], where the target density cannot be computed in closed-form. The noisy scenario also appears naturally when mini-batches of data are employed instead of considering the complete likelihood of huge amounts of data [10, 11]. More recently, the analysis of noisy functions of densities is required in reinforcement learning (RL), especially in direct policy search which is an important branch of RL, with applications in robotics [12, 13]. The topic of inference in noisy settings (or where a function is known with a certain degree of uncertainty) is also of interest in the inverse problem literature, such as in the calibration of expensive computer codes [14, 15]. This is also the case when the construction of an emulator is considered, as a surrogate model [4, 16, 17].