However, it remains largely open to learn energy-based latent variable models (EBLVMs), exceptsomespecialcases. Thispaperpresents abi-levelscorematching (BiSM) method to learn EBLVMs with general structures by reformulating SM as a bilevel optimization problem. The higher level introduces a variational posterior of the latent variables and optimizes a modified SM objective, and the lower level optimizes the variational posterior to fit the true posterior.

We thank all reviewers for their valuable comments. Below, we first address the common concerns and then answer the detailed questions. It leads to smaller bias (see Fig. A), which also agrees with Thm. 2. First, introducing latent variables can improve the sample quality (w.r.t. Indeed, we update Tab. 2 and obtain Tab. As stated in L290, a similar protocol is adopted in MDSM [34].