Reviews: SCAN: A Scalable Neural Networks Framework Towards Compact and Efficient Models

It is interesting to realize scalable neural networks in an architecture by introducing shallow classifiers. Actually, the motivation is not new as some recent work also investigated a similar objective [C1, C2, C3] by the anytime property. However, there are no analyses and comparison with the related studies which should be introduced and compared. The proposed framework is not that significant as the additional components are just borrowed from existing well-developed studies (attention, distillation) as well as the framework requires more parameters and computations. So the methodology itself is still incremental.