This paper addresses the problem of representing unordered point sets for recognition applications. The key insights is a "chi-convolution" operator that learns to "permute" local points and point-features into a canonical order within a neural network. The approach is demonstrated on 3D point cloud recognition and segmentation and 2D sketch and image classification applications. Positives: The paper addresses a known hard problem - how to properly represent unordered point sets for recognition. As far as I'm aware, the paper describes a novel and interesting approach for learning to "permute" local point neighborhoods in unordered point sets.

This work explores the applications of deep learning architectures with 3D geometric data such as point clouds or meshes. Usually, CNNs require regular data formats like image grids or 3D voxels. However, point clouds and meshes do not match the conventional format; therefore, people used to transform them into regular 3D voxel grids or collections of images. Although you could vaxolize the input data, the approach is very expensive since it results in a large input space (your mesh would be really sparse). PointNet, however, simply utilizes point clouds as input data and outputs either class labels for the entire input or for each individual point to perform Part Segmentation.

Learning the physical dynamics of deformable objects with particle-based representation has been the objective of many computational models in machine learning. While several state-of-the-art models have achieved this objective in simulated environments, most existing models impose a precondition, such that the input is a sequence of ordered point sets. That is, the order of the points in each point set must be the same across the entire input sequence. This precondition restrains the model from generalizing to real-world data, which is considered to be a sequence of unordered point sets. In this paper, we propose a model named time-wise PointNet (TP-Net) that solves this problem by directly consuming a sequence of unordered point sets to infer the future state of a deformable object with particle-based representation. Our model consists of a shared feature extractor that extracts global features from each input point set in parallel and a prediction network that aggregates and reasons on these features for future prediction. The key concept of our approach is that we use global features rather than local features to achieve invariance to input permutations and ensure the stability and scalability of our model. Experiments demonstrate that our model achieves state-of-the-art performance with real-time prediction speed in both synthetic dataset and real-world dataset. In addition, we provide quantitative and qualitative analysis on why our approach is more effective and efficient than existing approaches.