At thet-th backward propagation step, we can derive the gradient il(wt)toupdatei-th module inM. The number in the bracket represents the batch size. We see that when the batch size is small (i.e.,32), the gradientvariancesaresimilar. N and K indicate the number of FPN levels and region proposals fed into the detection head. To evaluate this assumption, as shown in Figure 1, we have three observations. As illustrated by the second figure in Figure 1, the gradient misalignment phenomenon between detection head and backbone has been reduced.

Table A1: 3D semantic segmentation results on the SemanticKiTTI validation set. We implemented our method with Pytorch using the open-source OpenPCDet [1]. The faded strategy was used during the last 5 epochs. It provides 22 sequences with 19 semantic classes, captured by a 64-beam LiDAR sensor. The 4th and 5th models sequentially incorporate our proposed SED blocks and DED blocks. Center-based 3d object detection and tracking.

Neural Information Processing Systems http://nips.cc/

Training a large-scale deep neural network in a large-scale dataset is challenging and time-consuming. The recent breakthrough of large-batch optimization is a promising way to tackle this challenge.

The essence of object detection lies in training a detection backbone to extract visual features from images and a detection head to recognize and locate objects based on the visual features.

There are a few recent studies on the topic, such as [33, 37].