First of all we would like to thank the Reviewers for their valuable comments and useful suggestions, which will be

Below, we address the main points raised by the Reviewers. "I'm not sure how well this is aligned with the goal of the paper" "I would have liked [...] evaluation of uncertainty calibration" "The paper does not compare with [...] deep ensembles [...] non-Bayesian "Why did you use SGHMC instead of full HMC?" But for this simple example we don't expect it to make big difference (the R-hat statistic showed its convergence). We will add some traces and the setup in the supplement. "Why do you assume a fully factorized Gaussian posterior Our parameterization in case of output dimension 1 will be equivalent to mean field.