"NIPS Neural Information Processing Systems 8-11th December 2014, Montreal, Canada",,, "Paper ID:","1711" "Title:","Distributed Bayesian Posterior Sampling via Moment Sharing" Current Reviews First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. This paper proposes a new distributed Bayesian Posterior inference algorithm for big data, an important problem that has garnered a lot of attention in the last few years. The goal is to sample from the posterior distribution of model parameters theta given a dataset that is divided among m machines. The proposed distributed algorithm runs a separate Markov chain on each machine, each of which samples from a distribution proportional to p(Dm|theta) qm(theta) where Dm is the local data subset on machine m and qm is a variational (Gaussian) approximation to the product of similar factors on other machines. I think this is a great idea and addresses a very important problem. The paper is also relatively easy to understand. However, the experimental section is quite weak which makes me a little hesitant to argue strongly for acceptance.

Neural Networks can perform poorly when the training label distribution is heavily imbalanced, as well as when the testing data differs from the training distribution. In order to deal with shift in the testing label distribution, which imbalance causes, we motivate the problem from the perspective of an optimal Bayes classifier and derive a post-training prior rebalancing technique that can be solved through a KL-divergence based optimization. This method allows a flexible post-training hyper-parameter to be efficiently tuned on a validation set and effectively modify the classifier margin to deal with this imbalance. We further combine this method with existing likelihood shift methods, re-interpreting them from the same Bayesian perspective, and demonstrating that our method can deal with both problems in a unified way.

Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

Macroscopic data aggregated from microscopic events are pe rvasive in machine learning, such as country-level COVID-19 infection statis tics based on city-level data.

We thank all reviewers for their comments and acknowledgeme nt of our contribution. Below we address each reviewer's comments separately. The reviewer raised a very good point. We will add this clarification in the revised version. Our gradient-based method is much more efficient but only finds a stationary point.