One of the fundamental aspects of over-parametrized models is that they are capable of interpolating the training data. We show that, under interpolation-like assumptions satisfied by the stochastic gradients in an overparametrization setting, thefirst-order oracle complexityofPerturbed Stochastic Gradient Descent (PSGD) algorithm toreach an -local-minimizer,matches the corresponding deterministic rateof O(1/2).

Then, we propose a series of DP-SignRP algorithms that leverage the robustness of the "sign flipping probability" of random projections.

In the setting of online learning, Implicit algorithms turn out to be highly successful from a practical standpoint.

In the setting of online learning, Implicit algorithms turn out to be highly successful from a practical standpoint.