Additional Feedback: On Reproducibility: - I think the basic methodology could be replicated, but it would have been nice to include code as a supplementary material. I hope the authors can assure me that the code will be documented and made available upon publication. On The Motor Cortex Dataset: - The VAE and pi-VAE seem to perform similarity in panel i and panel m - The better performance of pi-VAE in panel h vs l is likely due to the input variable "u" which forces different latent represenations (update: after writing this, I noticed that this is indeed the case based on supplementary figure S1; though pi-VAE is still slightly better). This is fine, but perhaps makes the result unsurprising -- wouldn't other supervised methods (e.g. On The Hippocampal Dataset: - For fig 4B, I think that linear discriminant analysis (LDA) would be sufficient to get you separation between the two running directions --- i.e. this would recover "latent 1".

Despite hinting at such a result multiple times in the paper, the results presented in this paper does not directly imply pure or approximate differential privacy for an algorithm that runs Langevin dynamics for T-iterations. At least it is not a trivial argument that goes through without further assumptions. The reason is the following: The Renyi Divergence bound on D(P R) (the order \alpha is abbreviated for readability) does not seem to imply a differential privacy bound overall, even though a sample from R satisfies DP. The DP bound of posterior sampling implies a bound on D(R R'). By results of this paper, we have bounds on D(P R) and D(P' R').

The reviewers agree that this paper provides an interesting analysis on the Langevin dynamics, which has interesting implications to differential privacy. The presentation is clear and the technical results are novel. The paper should clarify whether the finite-time variant of the dynamics actually leads to a private algorithm in their revision.

Weaknesses: Theorem 3.1 and its empirical results in Figure 1 bring the following concern: The fully connected graph obtained by lambda larger than certain value (e.g., lambda*), which is related to the input data covariance S. So, the scaling of X impacts lambda*. How does the lambda from 0 to lambda* theoretically impact the sparsity of the optimal graph? The proposed algorithm for solving nonconvex optimization is the combination of linearization of nonconvex regularization and projected gradient descent method. The convergence results are shown in Theorem 3.8. This can guarantee the convergence, but how many iterations are required for both inner and outer iterations in Algorithm 1 and what is the computational complexity?

Additional Feedback: I have read the other reviews and the authors' feedback. With the addition of the recommender system experiment, walking the readers through how (1) the MNAR and PPCA model apply in this setting, (2) selecting the hyper-parameters for the imputation algorithm, (3) showing how the imputations compare with prior algorithms, helps make a strong case for the proposed method. If the authors re-arrange the paper to improve clarity (as the reviews point out, and as they promise in their feedback), the paper can be substantially stronger. There are a few lingering questions from the reviews that the authors should address in the paper at a minimum -- (1) a discussion on a stage-wise approach to imputation (and why that may not be necessary for their sequence of regressions), (2) given that some of the linear coefficients can be zero, what must a practitioner do when one of the regressions estimate a coefficient close to 0 that is then used in the denominator of other estimates. Even better if the illustration is grounded in an example like movie item ratings.

There was no consensus among the referees for strongly supporting acceptance and it was felt that the paper was at-best marginal. We hope the reviews would be helpful in developing a stronger version of the paper.

This paper investigates the convergence of unadjusted Langevin algorithm (ULA) under the log-Sobolev inequality condition and shows convergence with respect to Reny divergence, which generalizes the existing results for KL-divergence. The analysis for the convergence on the Reny divergence is novel and interesting. However, Theorems 1 and 2 require some more discussions about comparison with existing studies. Actually, reviewer 3 pointed out the following paper: [3] Ma, Yi-An, et al. "Is There an Analog of Nesterov Acceleration for MCMC?." arXiv preprint arXiv:1902.00996 This paper shows the exponential convergence under the log-Sobolev inequality in its Sec.

Originality: First-order Gradient-based MCMC methods have to deal with determining an appropriate length scale for each variable. NUTS is one approach and this paper gives another approach whereby a parameter theta of a proposal distribution is adaptively improved to account for the covariance structure. At the same time theta is adapted to consider the entropy of the proposal distribution. This trade off for theta is rolled into a new speed measure which is the central point of this paper. The paper includes a lower bound of the speed measure that can be directly differentiated resulting in a practical algorithm. The paper also includes a heuristic that makes this adaptive MCMC algorithm applicable to MALA as well.

Proposes methods to update proposal distributions for MCMC using variational-type bounds and showed it is competitive with state of the art MCMC methods. Reviewers agreed this was an important contribution.

LDAM aims to put regularization on the margins (i.e. the minimum distance of data samples to the decision boundary) of minority classes in order to improve the generalizability of the model towards minority classes during the test time, in which the value of margin is set to be proportional to the number of samples for each class thus the LDAM is label-distribution-aware. DRW runs reweighting and LDAM with smaller learning rate in order to perform fine-tuning on the model after an initial stage of training. Although without any theoretical justification, its efficacy is successfully proven across various experiments. These ideas are novel and shown to provide better superior performance, even avoid overfitting for frequent classes, in comparison to naive re-weighting/re-sampling techniques and other baselines (e.g. It is also required to have an analysis on the sensitivity of performance with respect to C. Additionally, as in both stages of Algorithm.1 LDAM is used, should there be different values of C? - How is the LDAM-HG-DRS in Table.1 implemented?