LMC can suffer from slow convergence - requiring a large number of steps of small step-size to obtain good quality samples.

Throughout the appendix, we shall adopt an orthogonal parametrization for mean vectors in the modelM.

In this section, we present the training algorithm for Self-Tuning Networks. The full algorithm is showninAlg.2.

So we can check that ddtE(qt,λt) (qt,λt) in both cases. Combing the two cases yield the result. Pm i=1N(θ;µi,σ2i) where m is fixed to5 in all the experiments. Monotonic Bayesian Neural Networks In this experiment, we use the COMPAS dataset (J. The task istopredict whether the individual will commit acrime againin2years.

Yet, their method calledSTORM crucially relies on the knowledge of thesmoothness, aswellaboundonthegradient norms.

Additionally, it is evident that the results in (ii) do not implytheconvergenceof f(xk) to0.

In our method and all the baselines except surrogate-based triage, we use the cross-entropy loss and implement SGD using Adam optimizer [40] with initial learning rate set by cross validation independently foreachmethod andleveloftriageb. Insurrogate-based triage, weusethelossand optimization method used by the authors in their public implementation. Moreover, we use early stopping with the patience parameterep = 10,i.e.,we stop the training process ifno reduction of cross entropy loss is observed on the validation set. This suggests that the humans aremore accurate than thepredictivemodel throughout theentire feature space. This suggests that the humans are less accurate than the predictive model in some regions of the featurespace.

LMC can suffer from slow convergence - requiring a large number of steps of small step-size to obtain good quality samples.

Langevin Dynamics is a Stochastic Differential Equation (SDE) central to sampling and generative modeling and is implemented via time discretization. Langevin Monte Carlo (LMC), based on the Euler-Maruyama discretization, is the simplest and most studied algorithm. LMC can suffer from slow convergence - requiring a large number of steps of small step-size to obtain good quality samples. This becomes stark in the case of diffusion models where a large number of steps gives the best samples, but the quality degrades rapidly with smaller number of steps. Randomized Midpoint Method has been recently proposed as a better discretization of Langevin dynamics for sampling from strongly log-concave distributions. However, important applications such as diffusion models involve non-log concave densities and contain time varying drift. We propose its variant, the Poisson Midpoint Method, which approximates a small step-size LMC with large step-sizes. We prove that this can obtain a quadratic speed up of LMC under very weak assumptions. We apply our method to diffusion models for image generation and show that it maintains the quality of DDPM with 1000 neural network calls with just 50-80 neural network calls and outperforms ODE based methods with similar compute.