Model Compression using Progressive Channel Pruning

--In this work, we propose a simple but effective channel pruning framework called Progressive Channel Pruning (PCP) to accelerate Convolutional Neural Networks (CNNs). In contrast to the existing channel pruning methods that prune channels only once per layer in a layer-by-layer fashion, our new progressive framework iteratively prunes a small number of channels from several selected layers, which consists of a three-step attempting-selecting-pruning pipeline in each iteration. In the attempting step, we attempt to prune a pre-defined number of channels from one layer by using any existing channel pruning methods and estimate the accuracy drop for this layer based on the labelled samples in the validation set. In the selecting step, based on the estimated accuracy drops for all layers, we propose a greedy strategy to automatically select a set of layers that will lead to less overall accuracy drop after pruning these layers. In the pruning step, we prune a small number of channels from these selected layers. We further extend our PCP framework to prune channels for the deep transfer learning methods like Domain Adversarial Neural Network (DANN), in which we effectively reduce the data distribution mismatch in the channel pruning process by using both labelled samples from the source domain and pseudo-labelled samples from the target domain. Our comprehensive experiments on two benchmark datasets demonstrate that our PCP framework outperforms the existing channel pruning approaches under both supervised learning and transfer learning settings. HILE deep learning technologies have been successfully used for many computer vision tasks, it is still a challenging task to deploy deep neural networks on mobile devices due to tight computation resources and limited battery power. Several model compression approaches (see Section II for more details) have been recently developed to deploy deep models on resource-constrained devices, among which channel pruning technologies are attracting increasing attention as these technologies are often efficient on both CPUs and GPUs without requiring special implementation. In this work, we propose a new iterative channel pruning framework called Progressive Channel Pruning (PCP) for model compression under both supervised and transfer learning settings. Jinyang Guo, Weichen Zhang, Wanli Ouyang and Dong Xu are with the School of Electrical and Information Engineering, University of Sydney, Sydney, NSW, 2008 Australia.