Reviews: Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations

The proposed scene representation is a map phi from the 3D physical space to a feature space encoding properties such as color, distance from closest scene surface, in practice implemented with an MLP. This choice of parametrization results in a natural way of controlling the level of spatial detail the map can achieve with the chosen network capacity, without using a fixed/discrete spatial resolution (as in voxel grids). Images are generated from the scene representation phi, conditioned on a given camara (inclusive of intrinsic and extrinsic parameters) via a differentiable ray marching algorithm. Ray marching is performed using a fixed length unroll of an RNN which can operate on phi effectively decoding distance from the closest surface and learning to correctly how to update the marcher step length. This formulation has the nice bi-product of producing depth maps'for free'.