Explaining the effects of non-convergent sampling in the training of Energy-Based Models

In this paper, we quantify the impact of using nonconvergent Markov chains to train Energy-Based EBMs offer several fundamental advantages over their competitors models (EBMs). In particular, we show analytically due to their simplicity: A single neural network is that EBMs trained with non-persistent short involved in training, which means that fewer parameters runs to estimate the gradient can perfectly reproduce need to be learned and training is less costly. They are also a set of empirical statistics of the data, not at particularly appealing for interpretive applications: Once the level of the equilibrium measure, but through trained, the energy function can be analyzed with statistical a precise dynamical process. Our results provide a mechanics tools (Decelle & Furtlehner, 2021b), or shallow first-principles explanation for the observations of EBMs can serve as an effective model to "learn" something recent works proposing the strategy of using short from the data. EBMs have been exploited for instance to runs starting from random initial conditions as an infer the three dimensional structure (Morcos et al., 2011) efficient way to generate high-quality samples in or building blocks (Tubiana et al., 2019) of proteins, to generate EBMs, and lay the groundwork for using EBMs artificial pieces of genome (Yelmen et al., 2021), for as diffusion models. After explaining this effect in neuroimaging (Hjelm et al., 2014), simulation of complex generic EBMs, we analyze two solvable models in wavefunctions in quantum many-body physics (Carleo & which the effect of the non-convergent sampling Troyer, 2017; Melko et al., 2019), or to impute missing in the trained parameters can be described in detail.