Review for NeurIPS paper: Learning Latent Space Energy-Based Prior Model

Neural Information Processing Systems 

Weaknesses: * Missing comparison against persistent sampling: This paper proposes to use short-run MCMC to sample from both the prior and true posterior. In practice, since we have only one prior distribution, sampling from the prior can be also done using persistent sampling which often improves the performance of EBMs by a large margin. It's not clear why the proposed method uses short-run MCMC that can potentially mix slowly and can introduce sampling error. Moreover, Eq. 13 shows that sampling error turns the objective into an upper bound on the log-likelihood. This can be dangerous as the model may start increasing the gap between the distribution of approximate samples and the EBM prior by making the distribution harder to sample from.