However,learningefficiencyon these models should be carefully considered as the huge computation overhead.

A large part of the current success of deep learning lies in the effectiveness of data - more precisely: labelled data. Yet, labelling a dataset with human annotation continues to carry high costs, especially for videos. While in the image domain, recent methods have allowed to generate meaningful (pseudo-) labels for unlabelled datasets without supervision, this development is missing for the video domain where learning feature representations is the current focus.

First, let us recall that AIS is based on a simulated annealing process where a configuration is gradually brought from temperature T = to T = 1 using a set of bridging distributions. Foreach temperature, we define the transition operator, Tk(v0,v) to bring a configuration v to v0 varying the temperature according to the temperature schedule. In our case it is done using MC sampling layer-wise. In our work, we used a set of Nβ [104,105] temperatures uniformly distributed in this interval (dependingonthesystemsize). Inpractice,oneobservesthatERBM goesbelowED atlong sampling times if the machine was trained out of equilibrium.

Byanalyzing Local SGDA under the ideal condition of no gradient noise, we show that generally it cannot guarantee exact convergence with constant stepsizes and thus suffers from slow rates of convergence.