Interpretable Convolutional Filter Pruning

As significant redundancies inevitably present in such a structure, many works have been proposed to prune the convolutional filters for computation cost reduction. Although extremely effective, most works are based only on quantitative characteristics of the convolutional filters, and highly overlook the qualitative interpretation of individual filter's specific functionality. In this work, we interpreted the functionality and redundancy of the convolutional filters from different perspectives, and proposed a functionality-oriented filter pruning method. With extensive experiment results, we proved the convolutional filters' qualitative significance regardless of magnitude, demonstrated significant neural network redundancy due to repetitive filter functions, and analyzed the filter functionality defection under inappropriate retraining process. Such an interpretable pruning approach not only offers outstanding computation cost optimization over previous filter pruning methods, but also interprets filter pruning process. 1 I NTRODUCTION The great success of Convolutional Neural Network (CNN) is benefited from its advanced algorithm and architecture, which utilize interconnected multi-layer network structure to hierarchically abstract the data feature for recognition tasks (Krizhevsky et al. (2012)). However, this complex mechanism offers CNN outstanding performance at the price of intensive computation cost. Therefore, many CNN optimization works have been proposed to alleviate this computation cost (Han et al. (2015), Jaderberg et al. (2014), etc).