Efficient Combination of Rematerialization and Offloading for Training DNNs

Rematerialization and offloading are two well known strategies to save memory during the training phase of deep neural networks, allowing data scientists to consider larger models, batch sizes or higher resolution data.