A Details of Feature Extractor Adaptation

Therefore, we need to specialize the feature extractor to best match the target dataset. The peak memory cost of this phase is 61MB under resolution 224, which is reached when the largest sub-network is sampled. MAC (only forward) of sampled sub-nets is (355M + 1182M) / 2 = 768.5M Therefore, the total MAC of this phase is 768.5M Flowers, where 2040 is the number of total training samples, 0.2 means the validation set consists of Details of the accuracy predictor is provided in Appendix B. It takes the one-hot encoding of the sub-network's MAC of this accuracy predictor is only 0.37M, which is 3-4 orders of magnitude smaller than the Therefore, TinyTL is not only more memory-efficient but also more computation-efficient.