Reviews: Shadowing Properties of Optimization Algorithms
–Neural Information Processing Systems
The paper presents a theoretical analysis of how well a discrete dynamic flow approximates the flow/solution of a corresponding ODE for gradient descent and heavy ball methods, e.g., how trajectory of the discrete method with small enough step-size does not deviate too much from the trajectory of the ODE. The main theoretical results are somewhat limited, i.e., small step size and quadratic functinos, but are of interest.
Neural Information Processing Systems
Jan-25-2025, 06:29:39 GMT
- Technology: