A Survey on Deep Generative 3D-aware Image Synthesis

Despite achieving compelling results, most approaches focus on two-dimensional (2D) images, overlooking the three-dimensional (3D) nature of the physical world. The lack of 3D structure, therefore, inevitably limits some of their practical applications. Recent studies have thus proposed generative models that are 3D-aware. That is, they incorporate 3D information into the generative models to enhance control (especially in terms of multiconsistency) over the generated images. Examples depicted in Figure 1 elucidate that the objective is to produce high-quality renderings which maintain consistency across various views. In contrast to the 2D generative models, the recently developed 3D-aware generative models [13, 33] bridge the gap between 2D images and 3D physical world. The physical world surrounding us is intrinsically three-dimensional and images depict reality under certain conditions of geometry, material, and illumination, making it natural to model the image generation process in 3D spaces.