Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification - supplementary material Francesca Mignacco

The derivation of the self-consistent stochastic process discussed in the main text can be obtained using tools of statistical physics of disordered systems. In particular, it has been done very recently for a related model, the spherical perceptron with random labels, in [1]. Our derivation extends the known DMFT equations by including structure in the data; a stochastic version of gradient descent as discussed in the main text; the relaxation of the spherical constraint over the weights and the introduction of a Ridge regularization term. There are at least two ways to write the DMFT equations. One is by using field theoretical techniques; otherwise one can employ a dynamical version of the so-called cavity method [2].