This paper proposes a reliable neural network pruning algorithm by setting up a scientific control. Existing pruning methods have developed various hypotheses to approximate the importance of filters to the network and then execute filter pruning accordingly. To increase the reliability of the results, we prefer to have a more rigorous research design by including a scientific control group as an essential part to minimize the effect of all factors except the association between the filter and expected network output. Acting as a control group, knockoff feature is generated to mimic the feature map produced by the network filter, but they are conditionally independent of the example label given the real feature map. We theoretically suggest that the knockoff condition can be approximately preserved given the information propagation of network layers. Besides the real feature map on an intermediate layer, the corresponding knockoff feature is brought in as another auxiliary input signal for the subsequent layers. Redundant filters can be discovered in the adversarial process of different features. Through experiments, we demonstrate the superiority of the proposed algorithm over state-of-the-art methods. For example, our method can reduce 57.8% parameters and 60.2% FLOPs of ResNet-101 with only 0.01% top-1 accuracy loss on ImageNet.

Through standard Schur complement calculation, the semi-definite condition can be derived, i.e., The knockoff data are generated by the generator and then sent to the discriminator to verify whether the knockoff condition (Definition 1) holds. The distribution of features w.r .t. samples are shown in Figure S1, and 10K samples are sampled from ImagNet dataset.

Summary and Contributions: Post Rebuttal I have read the reviews of the fellow reviewers and the response of the authors. They have addressed the 2/3 of my points. Methodology and communication is somewhat lacking, and prevents a thorough understanding of the approach and implementation (acknowledged by R4 as well). The authors have agreed to work on this, although it is not sure how they plan to go about it. The results for the methods compared were changing between tables, and it was suggested to present mean and std.

This paper presents a method to prune filters in convolutional neural networks by introducing a scientific control group of knockoff features to reduce the disturbance of irrelevant factors. The authors also analyze the knockoff condition theoretically and derive the knockoff features given the knockoff data. Experiments are performed on CIFAR-10 and ImageNet. The reviewers and AC have read the author feedback carefully in addition to all the reviews. It is generally agreed that the proposed method is novel and interesting in that there is no need to specify arbitrary thresholds and hyperparameters for pruning.

This paper proposes a reliable neural network pruning algorithm by setting up a scientific control. Existing pruning methods have developed various hypotheses to approximate the importance of filters to the network and then execute filter pruning accordingly. To increase the reliability of the results, we prefer to have a more rigorous research design by including a scientific control group as an essential part to minimize the effect of all factors except the association between the filter and expected network output. Acting as a control group, knockoff feature is generated to mimic the feature map produced by the network filter, but they are conditionally independent of the example label given the real feature map. We theoretically suggest that the knockoff condition can be approximately preserved given the information propagation of network layers.