Profiling checkpointing schedules in adjoint ST-AD

Section 4 discusses the information Source-transformation algorithmic differentiation (STthat we found most appropriate to guide the choice of AD) in its adjoint mode transforms a primal code that activated checkpoints, and an algorithm to extract this evaluates some original function into an adjoint code information at run-time by profiling execution of the adjoint that computes its gradient. It is well known [9] that code. Section 5 discusses implementation of this the most efficient implementation of the adjoint code profiling in an existing source-transformation AD tool, must progress backwards of the original computation, and section 6 applies it to two realistic test-cases taken progressively using values originating from the primal from the MITgcm code suite. We will show how the execution. The amount of values used grows linearly developer can achieve a significant performance gain by with the run time of the primal code and, since they are exploiting the profiling results. In section 7, we come used in the reverse of their production order, their management back to some limitations of our proposed approach and (data-flow reversal) is a key issue that requires discuss how they could be overcome, before concluding a delicate trade-off between storage and recomputation. in section 8. This work focuses on one particular setting, where data-flow reversal is primarily done through a stack 2 Our checkpointing model / setting and the memory cost of this stack is mitigated through In our setting, data-flow reversal is achieved by storing a classical storage/recomputation trade-off known as intermediate values of the primal execution.