GauSS-MI: Gaussian Splatting Shannon Mutual Information for Active 3D Reconstruction

School of Computing and Data Science, The University of Hong Kong, Hong Kong SAR, China Email: {yuhanxie, zyq507 }@connect.hku.hk, Abstract --This research tackles the challenge of real-time active view selection and uncertainty quantification on visual quality for active 3D reconstruction. Visual quality is a critical aspect of 3D reconstruction. Recent advancements such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have notably enhanced the image rendering quality of reconstruction models. Nonetheless, the efficient and effective acquisition of input images for reconstruction--specifically, the selection of the most informative viewpoint--remains an open challenge, which is crucial for active reconstruction. Existing studies have primarily focused on evaluating geometric completeness and exploring unobserved or unknown regions, without direct evaluation of the visual uncertainty within the reconstruction model. T o address this gap, this paper introduces a probabilistic model that quantifies visual uncertainty for each Gaussian. Leveraging Shannon Mutual Information, we formulate a criterion, Gaussian Splatting Shannon Mutual Information (GauSS-MI), for real-time assessment of visual mutual information from novel viewpoints, facilitating the selection of next best view. GauSS-MI is implemented within an active reconstruction system integrated with a view and motion planner . Extensive experiments across various simulated and real-world scenes showcase the superior visual quality and reconstruction efficiency performance of the proposed system. Recent advancements, such as Neural Radiance Field (NeRF)[26] and 3D Gaussian Splatting (3DGS)[19], have notably enhanced the visual quality of 3D reconstruction models. However, these techniques necessitate the prior acquisition of a significant number of images, which can be laborious, and the extensive sampling of viewpoints may result in redundancy. Consequently, a challenging issue arises in effectively and efficiently selecting the viewpoints for image capture, which is also a critical problem for active 3D reconstruction. To enhance the autonomy of robots and enable them to perform 3D reconstruction tasks in complex environments, there has been a growing focus on active 3D reconstruction in recent years [43, 17, 34].