Understanding the Role of Momentum in Stochastic Gradient Methods
Igor Gitman, Hunter Lang, Pengchuan Zhang, Lin Xiao
–Neural Information Processing Systems
Different variants ofmomentum, including heavyball momentum, Nesterov's accelerated gradient (NAG), and quasi-hyperbolic momentum (QHM), havedemonstrated success onvarious tasks. Our results are most closely related to the work of Mandt et al.[19]who use stationaryanalysis of SGD with momentum to perform approximateBayesianinference.
Neural Information Processing Systems
Feb-12-2026, 03:25:50 GMT
- Country:
- Asia > Russia (0.05)
- Europe
- North America
- Canada > British Columbia
- United States > Georgia
- Fulton County > Atlanta (0.04)
- Genre:
- Research Report > New Finding (0.48)
- Technology: