Research Guide: Pruning Techniques for Neural Networks

The authors of this paper propose a network pruning pipeline that allows for pruning from scratch. Based on experimentation with compression classification models on CIFAR10 and ImageNet datasets, the pipeline reduces pre-training overhead incurred while using normal pruning methods, and also increases the accuracy of the networks. Below is an illustration of the three stages involved in the traditional pruning process. This process involves pre-training, pruning, and fine-tuning. The pruning technique proposed in this paper involves building a pruning pipeline that can be learned from randomly initialized weights.