In this work, we tackle model efficiency by exploiting redundancy in the implicit structure of the building blocks of convolutional neural networks. We start our analysis by introducing a general definition of Composite Kernel structures that enable the execution of convolution operations in the form of efficient, scaled, sum-pooling components. As its special case, we propose Structured Convolutions and show that these allow decomposition of the convolution operation into a sum-pooling operation followed by a convolution with significantly lower complexity and fewer weights. We show how this decomposition can be applied to 2D and 3D kernels as well as the fully-connected layers. Furthermore, we present a Structural Regularization loss that promotes neural network layers to leverage on this desired structure in a way that, after training, they can be decomposed with negligible performance loss. By applying our method to a wide range of CNN architectures, we demonstrate'structured' versions of the ResNets that are up to 2x smaller and a new Structured-MobileNetV2 that is more efficient while staying within an accuracy loss of 1% on ImageNet and CIFAR-10 datasets. We also show similar structured versions of EfficientNet on ImageNet and HRNet architecture for semantic segmentation on the Cityscapes dataset. Our method performs equally well or superior in terms of the complexity reduction in comparison to the existing tensor decomposition and channel pruning methods.

The easiest of these three attributes is padding . Dilated or atrous convolutions are prominent in semantic segmentation architectures. Hence, it is important to consider how we can decompose dilated structured convolutions. Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc. 34th Conference on Neural Information Processing Systems (NeurIPS 2020), V ancouver, Canada. Bottom shows the equivalent operation using sum-pooling.

Weaknesses: The major weaknesses of this paper are the soundness and significance. In terms of soundness, the biggest issue is the way the empirical evaluation was designed and executed. Firstly, it's extremely odd that the authors decided to implement the structured convolution as a post-training conversion, rather than directly implementing the structured convolution as a trainable architectural feature (e.g. Otherwise, the authors should demonstrate the feasibility of performing Eq (3) as quick fine-tuning for fully trained (regular) networks to justify their approach. Secondly, although the reduction in numbers of operations and parameters is overall impressive, the decrease in accuracies compared to baselines (Table 1 to 5) is also non-negligible, especially for more recent networks like EfficientNet and MobileNetV2.

Following discussion the reviewers are generally positive, and the rebuttal has addressed some of their initial concerns. Overall I think this will make a good poster. The main novelty is the introduction of a useful form of structured convolutions and a demonstration of its applicability for creating efficient neural networks. The results are strong, showing reduction of complexity with comparable performance in terms of accuracy.

In this work, we tackle model efficiency by exploiting redundancy in the implicit structure of the building blocks of convolutional neural networks. We start our analysis by introducing a general definition of Composite Kernel structures that enable the execution of convolution operations in the form of efficient, scaled, sum-pooling components. As its special case, we propose Structured Convolutions and show that these allow decomposition of the convolution operation into a sum-pooling operation followed by a convolution with significantly lower complexity and fewer weights. We show how this decomposition can be applied to 2D and 3D kernels as well as the fully-connected layers. Furthermore, we present a Structural Regularization loss that promotes neural network layers to leverage on this desired structure in a way that, after training, they can be decomposed with negligible performance loss.