Consensus Monte Carlo (CMC) is a method for parallelizing MCMC for posterior inference over large datasets. It works by factorizing the posterior distribution into sub-posteriors each of which depend on only a subset of datapoints, sampling from each of these sub-posteriors in parallel, and then transforming samples from the sub-posteriors using an aggregation function to samples from the real posterior. Existing works use very naive methods of aggregation which result in high bias, or are computationally very expensive, which make it difficult to use Consensus Monte Carlo in practice. This paper proposes a more principled way of combining samples by optimizing over aggregation functions using variational inference. Clarity: The paper is well written and easy to follow.

Additional Feedback: - Line 70: DB methods are based on Minkowski distance, however, in this paper the duality is stablished only for the case of Frobenius norm, i.e. Minkowski distance with p 2. It would be nice that authors provide a deeper explanation about the role on parameter p in DB methods. What is the optimal value of p in state of the arts methods? The sentence: "the regularizer 5 and 6" should be changed to "the regularizer 4 and 5" (check equation numbering) - Line 180: As a regularizer having several terms, it would be convenient to consider different regularizer coefficients as hyperparameters. In fact, in supplemental material (lines 36-37) the cost function has 3 hyperparameters: lambda, lambda_1 and lambda_2.

There are roughly two different approaches in the literature for knowledge graph completion (KGC), namely distance based (DB) models and tensor factorization based (TFB) models. Although both approaches have their own advantages and disadvantages over each other, TFB models cannot attain state-of-the-art performance due to overfitting problem, and therefore various regularizers are employed for TFB models. In the paper, authors propose a regularizer for TFB models, namely Duality-induced Regularization (DURA), which is inspired by the score functions of the DB models. They come up with a dual problem which involves a distance based KGC model, and show that when the aforementioned regularizer is employed for the primal problem (i.e. TFB model), both problems become equivalent.

Summary and Contributions: Reconstruction of a low-rank matrix from its linear measurements is a canonical problem in machine learning and signal processing. There has been an intense effort to establish theoretical guarantees and design efficient algorithms for solving these problems. Of these, the most prominent two methods are: 1- Convex optimization approach - Nuclear-norm regularization. In particular, the non-convex factorization approach has received increasing attention due to the reduced arithmetic and storage costs. Recently, Gunasekar et al. (2017) reported a surprising observation, that the non-convex factorization approach (when solved with gradient descent) generalizes (i.e., recovers the low-rank matrix of interest) even when the factors U and V are full dimensional (i.e., not tall, hence UV' does not impose an explicit low-rank structure).

Summary and Contributions: ##Update## The rebuttal adequately addressed my main concerns and I am consequently increasing my score to a 7. In particular I was pleased that the authors investigated the issues with the learning rate, and I would be happy if they mention this potential limitation in their revisions, and include the experimental results showing that the naive adaptive learning rate proposals I made would not be effective. It was also pleasing that they will discuss and compare with Neural Replicator Dynamics, and the additional experiment with sampled actions also looks promising. The reason I didn't increase my score further was that the current set of experiments is still rather simple, and it is difficult for me to assess whether the new method is likely to be widely used. Though, I feel that the contribution may well turn out to be much more influential.

This paper is proposing alternative to common practices in machine learning: Softmax Policy Gradient for RL and softmax parameterization in classification when minimizing cross-entropy loss. The limitation of softmax in these two cases are well explained, and the paper will be interesting for a wide range of the NeurIPS community.

Overview: This paper studies the benefits of augmenting the linear programming relaxation of the maximum a-posteriori (MAP) inference problem in graphical models with a quadratic term, thereby achieving strong convexity. Such augmented formulations are obtained both from the original primal and dual formulations, and in each case the resulting primal-dual relationship is studied. Prior work has mostly focused on smoothing the LP formulation using a softmax/entropy term, with a few notable exceptions, such as [5], [17] and [18]. Rather than those previous approaches, which employ a quadratic term in the sub-problems of either a *proximal* or a *alternating direction* scheme, in the present manuscript, the quadratic smoothing term is added directly. This can in some way be seen as a naive approach: In comparison to proximal or alternating direction schemes, convergence to the global optimum of the original problem is no longer guaranteed, and the approximation quality directly depends on the strength of the augmentation term.

Additional Feedback: Line 68: Kleindessner et al. designed an algorithm for k-center with different type of fairness requirement. Instead of balancing different colors in each cluster, the goal is to pick centers (proportionally) from different colors. It is basically k-center under partition matroid. Line 69-70: In a(n almost) concurrent work, the fair correlation was also studied by Ahamdi et al. Line 131: Bounded representation: with binary colors, it is the same as balance.

The paper considers hierarchical clustering with fairness constraints and show the concept of fairlet decomposition extends to fair hierarchical clustering as well. The notion of fairness is important though it would be good if the authors improve the motivation section of this work.

Summary of paper: The paper studies a class of asynchronous methods to solving optimization problems in which the objective function is an integral functional of a random process or variable integrated against a probability measure. The class of problem assumed is that the argument of the functional for each non-zero measure set or outcome in the sample space upon which the measure is defined will produce a convex function. In this case the authors demonstrate that asymptotically, completely asynchronous stochastic gradient procedures achieve optimal convergence rates for the solution of convex optimization problems under nearly the same conditions required for asymptotic optimality of standard stochastic gradient procedures. The authors are able to then propose based on these results specialised asynchronous procedures based on multi-core parallelization schemes for stochastic gradient based optimization algorithms which behave analogously to synchronous procedures. Quality: The problem statement and all assumptions are very carefully set out and clearly written and explained with regard to the meaning and significance of such assumptions.