Wedosobyintroducing apairofoperators: gather,which efficiently aggregates feature responses from a large spatial extent, and excite, which redistributes the pooled information to local features.

While the use of bottom-up local operators in convolutional neural networks (CNNs) matches well some of the statistics of natural images, it may also prevent such models from capturing contextual long-range feature interactions. In this work, we propose a simple, lightweight approach for better context exploitation in CNNs. We do so by introducing a pair of operators: gather, which efficiently aggregates feature responses from a large spatial extent, and excite, which redistributes the pooled information to local features. The operators are cheap, both in terms of number of added parameters and computational complexity, and can be integrated directly in existing architectures to improve their performance. Experiments on several datasets show that gather-excite can bring benefits comparable to increasing the depth of a CNN at a fraction of the cost. For example, we find ResNet-50 with gather-excite operators is able to outperform its 101-layer counterpart on ImageNet with no additional learnable parameters. We also propose a parametric gather-excite operator pair which yields further performance gains, relate it to the recently-introduced Squeeze-and-Excitation Networks, and analyse the effects of these changes to the CNN feature activation statistics.

This paper proposes a new component for residual networks to improve the performance on image classification task. The proposed idea is to find channel-specific attention from each feature maps, which is to gather the spatial information by a particular layer such as the average pooling, and then the scatter layer such as nearest neighbor upsampling is followed. Experiments are done on the datasets including ImageNet-1k and CIFAR-datasets, and the study of the class sensitivity compared with the original ResNets is provided to support the effectiveness of the proposed method. Pros)) The proposed spatial attention idea is simple yet promising. Cons)) Some notations are not clearly presented throughout the paper.

While the use of bottom-up local operators in convolutional neural networks (CNNs) matches well some of the statistics of natural images, it may also prevent such models from capturing contextual long-range feature interactions. In this work, we propose a simple, lightweight approach for better context exploitation in CNNs. We do so by introducing a pair of operators: gather, which efficiently aggregates feature responses from a large spatial extent, and excite, which redistributes the pooled information to local features. The operators are cheap, both in terms of number of added parameters and computational complexity, and can be integrated directly in existing architectures to improve their performance. Experiments on several datasets show that gather-excite can bring benefits comparable to increasing the depth of a CNN at a fraction of the cost.