We provide the definitions of important terms used throughout the paper. Assumption 2.3 when the demand distribution is exponential. Note that Lemma B.1 implies that In the following result, we show that there exist appropriate constants such that prior distribution satisfies Assumption 2.3 when the demand distribution is a multivariate Gaussian with unknown The proof is a direct consequence of Theorem 3.2, Lemmas B.6, B.7, B.8, B.9, and Proposition 3.2. Theorem 6.19] the prior induced by Assumption 2.2 is a direct consequence of Assumption 2.4 and 2.5 are straightforward to satisfy since the model risk function Lemma B.13. F or a given Using the result above together with Proposition 3.2 implies that the RSVB posterior converges at C.1 Alternative derivation of LCVB We present the alternative derivation of LCVB. We prove our main result after a series of important lemmas.

We compute the convergence rates of the RSVB approximate posterior and the corresponding optimal value.

The first termkxt x αk2 can be simply merged in eq. The first term is consensus error and will be merged inT1 in eq. Response: We have conducted more experiments over the ImageNet37 dataset which is known as a complicated dataset. As the middle figure38 demonstrates, for a binary classification, our method significantly im-39 proves upon the benchmarks over this dataset as well. We also carried40 out experiments on a deeper neural network with 4 hidden layers and41 ourmethod providessignificant speedups overthebenchmarks (bottom42 figure).

Assume the conditions in Proposition 1 hold.

In this section, we provide two different ways to prove Theorem 2. The first one is more straightforward and directly differentiates through the term To solve this issue, we introduce the following variational representation: Lemma 1. With Jensen's inequality, we have: log null null As introduced in Equation (9) in Section 2.3, the divergence corresponding to the This is a direct consequence of Lemma 2. It can also be verified by checking the PS-CD Lemma 3. When 1 γ < 0, we have: S We first make the following assumption, which is similar to the one used in [4, 47]: Assumption 1. The assumption is typically easy to enforce in practice. In this section, we analyze the convergence property of the PS-CD algorithm presented in Algorithm 1. We have the following theorem that characterizes the convergence property of Algorithm 2: Theorem 5. Monte Carlo estimation will incur additional approximation error.