A Pseudocode for K-BFGS/K-BFGS(L)

Algorithm 4 gives pseudocode for K-BFGS/K-BFGS(L), which is implemented in the experiments. In this section, we prove the convergence of Algorithm 5, a variant of K-BFGS(L). To accomplish this, we prove Lemmas 1-3, which in addition to Assumptions AS.1-2, ensure that all of the assumptions in Theorem 2.8 in [ Algorithm 6 SQN method for nonconvex stochastic optimization.Require: Given θ ( k 1) ( k 1) ( k 1) ( k 1) Hence, Theorem 2.8 of [41] applies to Algorithm 5, proving Theorem 2. Thus, we propose the following heuristic based on Powell's damped-BFGS approach In Powell's damping on H (see e.g. This is used in lines 2 and 3 of the DD (Algorithm 3). Our double damping (Algorithm 3) is a two-step damping procedure, where the first step (i.e.