Introducing PyTorch3D: An open-source library for 3D deep learning

But research in 3D deep learning has been limited because of the lack of sufficient tools and resources to support the complexities of using neural networks with 3D data and the fact that many traditional graphic operators are not differentiable. Facebook AI has built and is now releasing PyTorch3D, a highly modular and optimized library with unique capabilities designed to make 3D deep learning easier with PyTorch. PyTorch3D provides a set of frequently used 3D operators and loss functions for 3D data that are fast and differentiable, as well as a modular differentiable rendering API -- enabling researchers to import these functions into current state-of-the-art deep learning systems right away. PyTorch3D was recently a catalyst in Facebook AI's work to build Mesh R-CNN, which achieved full 3D object reconstruction from images of complex interior spaces. We fused PyTorch3D with our highly optimized 2D recognition library, Detectron2, to successfully push object understanding to the third dimension.