Appendix AAnalysisofvarianceofuncertaintyestimators

We sample 1,000 points at random in latent space from an isotropic Gaussian with fixed standard deviation σ (we repeat the experiment for different values of the standard deviation). We analyze the impact of the number ofys samples for each estimator on the variance of the corresponding estimators, measured over 10 independent runs. We use the sum of all pixel intensities in the image as a proxy of the thickness of the digits, which provides strongempiricalresults. Wejointly train avariational autoencoder with an auxiliary network (the "Property network") predicting digit thickness based on latent representation (see Figure 1). We evaluate the IS-MI estimator values in different regions of latent space.