Weakly Supervised Learning of Rigid 3D Scene Flow

We propose a data-driven scene flow estimation algorithm exploiting the observation that many 3D scenes can be explained by a collection of agents moving as rigid bodies. At the core of our method lies a deep architecture able to reason at the object-level by considering 3D scene flow in conjunction with other 3D tasks. This object level abstraction, enables us to relax the requirement for dense scene flow supervision with simpler binary background segmentation mask and ego-motion annotations. Our mild supervision requirements make our method well suited for recently released massive data collections for autonomous driving, which do not contain dense scene flow annotations. As output, Figure 1: Our network takes two successive frames as input our model provides low-level cues like pointwise flow (a), and outputs a set of transformation parameters for each and higher-level cues such as holistic scene understanding segmented rigid agent (c) which are used to recover perpoint at the level of rigid objects. We further propose a test-time rigid scene flow. After applying the predicted flow to optimization refining the predicted rigid scene flow. We the first point cloud, the two frames are aligned (b, d).