Supplementary Materials A Hessian Vector Implementation

We then select those that yield the best convergence performance. However, our code supports GPU cluster training. VRBO becomes slower and less stable. As a result, single-sample based algorithms enable a larger parameter update per sample, and hence achieve a higher sample efficiency. Besides, we apply the standard grid search for the inner-and outer-loop stepsizes for all algorithms.