Reviews: Shadowing Properties of Optimization Algorithms

Neural Information Processing Systems 

The paper presents a theoretical analysis of how well a discrete dynamic flow approximates the flow/solution of a corresponding ODE for gradient descent and heavy ball methods, e.g., how trajectory of the discrete method with small enough step-size does not deviate too much from the trajectory of the ODE. The main theoretical results are somewhat limited, i.e., small step size and quadratic functinos, but are of interest.