AppendixforBackRazor: Memory-EfficientTransfer LearningbySelf-SparsifiedBackpropagation

All the reported results are the top1 accuracy (%). Table 5: The on-device memory comparison between Back Razor and fully fine-tuning (test at a batchsizeof8inCIFAR100). Batchsize Baseline BackRazor@90% 8 1655MB 1637MB 128 7393MB 6634MB 3 Table 3: The comparison between Back Razor and TinyTL onFlowers with batch size of1. Actnn: Reducing training memory footprint via 2-bit activation compressed training.