Point Neighborhood Embeddings

Point convolution operations rely on different embedding mechanisms to encode the neighborhood information of each point in order to detect patterns in 3D space. However, as convolutions are usually evaluated as a whole, not much work has been done to investigate which is the ideal mechanism to encode such neighborhood information. In this paper, we provide the first extensive study that analyzes such Point Neighborhood Embeddings (PNE) alone in a controlled experimental setup. From our experiments, we derive a set of recommendations for PNE that can help to improve future designs of neural network architectures for point clouds. Our most surprising finding shows that the most commonly used embedding based on a Multi-layer Perceptron (MLP) with ReLU activation functions provides the lowest performance among all embeddings, even being surpassed on some tasks by a simple linear combination of the point coordinates. Additionally, we show that a neural network architecture using simple convolutions based on such embeddings is able to achieve state-of-the-art results on several tasks, outperforming recent and more complex operations. Lastly, we show that these findings extrapolate to other more complex convolution operations, where we show how following our recommendations we are able to improve recent state-of-the-art architectures. In computer vision, point clouds are one of the most commonly used representations to process and store 3D data. This is because point clouds are a compact representation and most 3D acquisition hardware produces point clouds as their output. In the past years, the advances in 3D acquisition hardware, and therefore the number of available point clouds, allowed the development of data-driven methods to solve different 3D scene understanding problems. In particular, the pioneering work of Qi et al. (2017a) opened the door to new neural network architectures which were able to process point clouds directly.