Supplementary Material: Appendices A Derivation of Variational System Let us consider a perturbed initial condition

Suppose that the solution x (t) satisfies x ( t) = x (t) + δ (t). Proof of Remark 4: Eq. (6) can be rewritten as λ The variational and adjoint variables are augmented in the same way. By definition, the naive backpropagation algorithm, baseline scheme, ACA, and the proposed symplectic adjoint method provide the exact gradient up to rounding error. The memory consumption and computation time depend highly on the implementations and devices. Being implemented on a GPU, the convolution operation can be easily parallelized in space and exhibits a non-deterministic behavior.