Code is available at this repository: https://github.com/thuml/Koopa.

The proposed template encompasses a wide range of popular first-order methods, including dual averaging, dual extrapolation andoptimistic gradient algorithms -both adaptive and non-adaptive. Our first result isthat thealgorithm achievestheoptimal rates of convergence for cocoercive problems when the profile of the randomness is known to the optimizer: O(1/ T) for absolute noise profiles, andO(1/T) for relative ones.