CATNIPS: Collision Avoidance Through Neural Implicit Probabilistic Scenes

Abstract--We introduce a transformation of a Neural Radiance Field (NeRF) to an equivalent Poisson Point Process (PPP). This PPP transformation allows for rigorous quantification of uncertainty in NeRFs, in particular, for computing collision probabilities for a robot navigating through a NeRF environment. The PPP is a generalization of a probabilistic occupancy grid to the continuous volume and is fundamental to the volumetric ray-tracing model underlying radiance fields. Building upon this PPP representation, we present a chance-constrained trajectory optimization method for safe robot navigation in NeRFs. Our method relies on a voxel representation called the Probabilistic Unsafe Robot Region (PURR) that spatially fuses the chance constraint with the NeRF model to facilitate fast trajectory optimization. We then combine a graph-based search with a spline-based trajectory optimization to yield robot trajectories through the NeRF that are guaranteed to satisfy a user-specific collision probability. We validate our chance constrained planning method through simulations and hardware experiments, showing superior performance compared to prior works on trajectory planning in NeRF environments. Figure 1: (a) Ground-truth of the Stonehenge scene, (b) Poisson Constructing an environment model from onboard sensors, Point Process (PPP) of the scene represented as a point cloud, such as RGB(-D) cameras, lidar, or touch sensors, is a fundamental (c) Probabilistically Unsafe Robot Region (PURR) of scene, challenge for any autonomous system. Radiance Fields (NeRFs) [1] have emerged as a promising 3D scene representation with potential applications in a variety of robotics domains including SLAM [2], pose estimation [3], such as (watertight) triangle meshes [9], occupancy grids [10], [4], reinforcement learning [5], and grasping [6]. NeRFs offer or Signed Distance Fields (SDFs) [11], occupancy is welldefined several potential benefits over traditional scene representations: and simple to query. NeRFs, however, do not admit they can be trained using only monocular RGB images, they simple point-wise occupancy queries, since they represent the provide a continuous representation of obstacle geometry, and scene geometry implicitly through a continuous volumetric they are memory-efficient, especially considering the photorealistic density field.