A Dynamical Mean-Field Theory for Learning in Restricted Boltzmann Machines

We define a message-passing algorithm for computing magnetization s in Restricted Boltzmann machines, which are Ising models on bipartite g raphs introduced as neural network models for probability distributions over spin con figurations. To model nontrivial statistical dependencies between the spins' couplings, we assume that the rectangular coupling matrix is drawn from an arbitrary bi-rotation in variant random matrix ensemble. Using the dynamical functional method of statist ical mechanics we exactly analyze the dynamics of the algorithm in the large system limit. We prove the global convergence of the algorithm under a stability criterion and c ompute asymptotic convergence rates showing excellent agreement with numerical sim ulations.