Reviewers reached consensus that the paper makes a valuable contribution for MCMC. There are specific suggestions for improving the experiments that we ask the authors to seriously consider.

The manuscript proposes a new objective function for learning Gaussian latent factor models. The objective function is based on information-theoretic characterization of modular latent factor models, where the model attains optimal value. The derivation of the objective function carefully avoids matrix inversion to improve computational complexity compared to traditional methods. The authors pointed out that the proposed model enjoys'blessing of dimension' in that model performance improves when the dimension of observable variables increases while the dimension of latent variables remains constant. This is demonstrated by both simulation and an information-theoretic lower bound on the sample size.

This manuscript proposes an estimator for graphical models which encourages modularity. The strengths of the manuscript include the conceptual simplicity of the proposal and the clear analysis. Reviewers also commented on the overall clarity of the presentation and the extensive experiments. I encourage the authors to read the reviews carefully and make changes as appropriate for the final version.

Major comments: Throughout the paper k is used to refer to the maximum number of edges in a graph, but I'm unclear if that means that the proofs hold for some k max(k_1,k_2) where k_1 is the number of edges in G_1, etc, and k_1 and k_2 are allowed to be different. If it doesn't allow each graph to have a different value for k this should be made clear. If it does allow that, then it's unclear what ranges of k the proofs hold for (presumably min(k_1,k_2) is lower bounded by \gamma k). Allowing each graph to have a different k implies that there could be different recovery rates for each graph, but the error metric is over the full joint space (rather than the subgraph or independently in the two graphs). Is it possible to make statements about the error metric just in the shared subgraph?

This paper studies the problem of "simultaneously learning two Ising models whose underlying graphs have some similarity constraints." The problem is interesting (and well-motivated) and the authors provide matching upper and lower bounds, with sharper characterization in some regimes. The proofs use more or less standard approaches, although applying these requires nontrivial work. Overall this is a solid contribution to NeurIPS. The authors responded to most of the reviewer's concerns.

Weaknesses: - Can we interpret the results as follows: If the TAR assumption is satisfied with positive limits, and we use MLE, then temporal interference does not cause bias. If this interpretation is correct, then it would be illuminating if the authors provide the intuitive connection between the TAR assumption and temporal interference. It is not clear if the estimations that the authors have required are feasible if the state space is large. The next natural question is how robust the results are if we use other methods for estimation. This could have been shown by providing some simulations, which is a part missing from the manuscript.

The paper studied the online experimental design problem where there are temporal dependencies between the two control policies/treatments. The novelty of the problem setup and the theoretical analysis in the paper are appreciated by all the reviewers. Although the analysis is the main contribution, the paper would be much stronger if there are meaningful experiments on toy problems to showcase the performance the online MLE-based approach vs the standard experimental design approaches.

This paper contributes a new technique for the estimation of structure in continuous time Bayesian networks, and completes the picture with an accompanying inference method and an illustration on a real-world problem. There is agreement among reviewers that this is a high quality contribution, if one takes the confidence-weighted scores from reviewers into account. As a point for improvement for the paper, we could reiterate a comment that was raised in the reviewer discussion: "[the paper] is missing reasonable and helpful experimental comparisons that are not hard to do, given that the code exists already in CTBN-RLE" and the authors are encouraged to consider broadening their experimental comparisons for a final published version.

Implementations of the exponential mechanism in differential privacy often require sampling from intractable distributions. When approximate procedures like Markov chain Monte Carlo (MCMC) are used, the end result incurs costs to both privacy and accuracy. Existing work has examined these effects asymptotically, but implementable finite sample results are needed in practice so that users can specify privacy budgets in advance and implement samplers with exact privacy guarantees. In this paper, we use tools from ergodic theory and perfect simulation to design exact finite runtime sampling algorithms for the exponential mechanism by introducing an intermediate modified target distribution using artificial atoms. We propose an additional modification of this sampling algorithm that maintains its \epsilon -DP guarantee and has improved runtime at the cost of some utility.

Variational autoencoders (VAEs) are a popular framework for modeling complex data distributions; they can be efficiently trained via variational inference by maximizing the evidence lower bound (ELBO), at the expense of a gap to the exact (log-)marginal likelihood. While VAEs are commonly used for representation learning, it is unclear why ELBO maximization would yield useful representations, since unregularized maximum likelihood estimation cannot invert the data-generating process. Yet, VAEs often succeed at this task. We seek to elucidate this apparent paradox by studying nonlinear VAEs in the limit of near-deterministic decoders. We first prove that, in this regime, the optimal encoder approximately inverts the decoder---a commonly used but unproven conjecture---which we refer to as self-consistency.