Gradient Descent
Adaptive Proximal Gradient Method for Convex Optimization
In this paper, we explore two fundamental first-order algorithms in convex optimization, namely, gradient descent (GD) and proximal gradient method (ProxGD). Our focus is on making these algorithms entirely adaptive by leveraging local curvature information of smooth functions.
Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning
Machine unlearning has raised significant interest with the adoption of laws ensuring the "right to be forgotten".
The Implicit Bias of Gradient Descent toward Collaboration between Layers: A Dynamic Analysis of Multilayer Perceptions
Another intriguing phenomenon is the existence of adversarial examples -- imperceptible perturbations of inputs that alter classification results.
Small steps no more: Global convergence of stochastic gradient bandits for arbitrary learning rates
In particular, we establish the surprising result that: F or any constant learning rate ฮท > 0, the stochastic gradient bandit algorithm is guaranteed to converge to the globally optimal policy almost surely.
The Sample Complexity of Gradient Descent in Stochastic Convex Optimization
But, also, it has become focus of study because it is one of few theoretical settings that exhibit overparameterized learning .
Gradient-based Discrete Sampling with Automatic Cyclical Scheduling
We prove the non-asymptotic convergence and inference guarantee for our method in general discrete distributions.
Private Geometric Median Mahdi Haghifam Thomas Steinke Jonathan Ullman
The predominant algorithm for DP convex optimization is DP (stochastic) gradient descent, or DP-(S)GD, for short [SCS13; BST14].