The Feature Pyramid Network (FPN) presents a remarkable approach to alleviate the scale variance in object representation by performing instance-level assignments. Nevertheless, this strategy ignores the distinct characteristics of different sub-regions in an instance. To this end, we propose a fine-grained dynamic head to conditionally select a pixel-level combination of FPN features from different scales for each instance, which further releases the ability of multi-scale feature representation. Moreover, we design a spatial gate with the new activation function to reduce computational complexity dramatically through spatially sparse convolutions. Extensive experiments demonstrate the effectiveness and efficiency of the proposed method on several state-of-the-art detection benchmarks.

Extensive experiments demonstrate the effectiveness and efficiency of the proposed method on several state-of-the-art detection benchmarks.

Summary and Contributions: The paper proposes a novel fine-grained dynamic routing mechanism for object detection where it conditionally selects features from multiple FPN scales for each pixel-level sub-region of an instance by using the data-dependent spatial gates. To reduce computational complexity, spatially sparse convolution is used in combination with a newly proposed activation function based on restricted tanh. Depth path uses bottleneck module with residual connection. Scale paths use common topology except for the bilinear operation. Output of the spatial gate is the gating factor, which is continuous, representing an estimate of probability of path being enabled.

The idea of conditionally merging features from different scales in FPN is interesting as acknowledged by all reviewers, and promising results are obtained by the proposed method. The AC agrees with the reviewers that the paper passes the acceptance bar of NeurIPS and recommends acceptance. The authors should add the clarifications and discussion in the rebuttal to the camera-ready version.

The Feature Pyramid Network (FPN) presents a remarkable approach to alleviate the scale variance in object representation by performing instance-level assignments. Nevertheless, this strategy ignores the distinct characteristics of different sub-regions in an instance. To this end, we propose a fine-grained dynamic head to conditionally select a pixel-level combination of FPN features from different scales for each instance, which further releases the ability of multi-scale feature representation. Moreover, we design a spatial gate with the new activation function to reduce computational complexity dramatically through spatially sparse convolutions. Extensive experiments demonstrate the effectiveness and efficiency of the proposed method on several state-of-the-art detection benchmarks.

The Feature Pyramid Network (FPN) presents a remarkable approach to alleviate the scale variance in object representation by performing instance-level assignments. Nevertheless, this strategy ignores the distinct characteristics of different sub-regions in an instance. To this end, we propose a fine-grained dynamic head to conditionally select a pixel-level combination of FPN features from different scales for each instance, which further releases the ability of multi-scale feature representation. Moreover, we design a spatial gate with the new activation function to reduce computational complexity dramatically through spatially sparse convolutions. Extensive experiments demonstrate the effectiveness and efficiency of the proposed method on several state-of-the-art detection benchmarks.