Gradient Descent
Federated
Algorithm 1Federated Accelerated Stochastic Gradient Descent ( FEDAC) 1: procedureFEDAC( , , , ).See Eqs. Gregory Francis Coppola.Iterative Parameter Mixingfor Distributed Large-Margin Trainingof Structured Predictorsfor Natural Language Processing.
Projection-Free Methods for Stochastic Simple Bilevel Optimization with Convex Lower-level Problem
However, all the above results are limited to the deterministic setting.
Preconditioned Gradient Descent for Over-Parameterized Nonconvex Matrix Factorization
In fact, a good damping parameter can be inexpensively estimated from the current iterate.
Stochastic Optimization for Performative Prediction
What sets this setting apart from traditional stochastic optimization is the difference between merely updating model parameters and deploying the new model.
