OntheAlmostSureConvergenceofStochastic GradientDescentinNon-ConvexProblems

We first showthat the sequence ofiterates generated bySGDremains bounded and converges with probability1 under a very broad range of step-size schedules. Subsequently, going beyond existing positive probability guarantees, we show that SGD avoids strict saddle points/manifolds with probability1 for the entire spectrum ofstep-size policies considered.