Goto

Collaborating Authors

 Technology


Benchmarking and Analyzing 3D-aware Image Synthesis with a Modularized Codebase

Neural Information Processing Systems

Despite the rapid advance of 3D-aware image synthesis, existing studies usually adopt a mixture of techniques and tricks, leaving it unclear how each part contributes to the final performance in terms of generality. Following the most popular and effective paradigm in this field, which incorporates a neural radiance field (NeRF) into the generator of a generative adversarial network (GAN), we build a well-structured codebase, dubbed Carver, through modularizing the generation process. Such a design allows researchers to develop and replace each module independently, and hence offers an opportunity to fairly compare various approaches and recognize their contributions from the module perspective. The reproduction of a range of cutting-edge algorithms demonstrates the availability of our modularized codebase. We also perform a variety of in-depth analyses, such as the comparison across different types of point feature, the necessity of the tailing upsampler in the generator, the reliance on the camera pose prior, etc., which deepen our understanding of existing methods and point out some further directions of the research work. We release code and models here to facilitate the development and evaluation of this field.




Appendix Meta-Learning with Self-Improving Momentum Target AOverview of terminologies used in the paper

Neural Information Processing Systems

The meta-learner network, i.e., learns to generalize on new tasks. Algorithm for adapting the meta-model into a task expert by using a given task dataset. Support set S. A dataset sampled from a given task distribution that is used for the adaptation. Query set Q. A dataset sampled from a given task distribution (that is disjoint with the support set) to evaluate the adaptation performance of the algorithm. Network adapted from the meta-model using the support set by using the adaptation subroutine, i.e., Adapt(,S) Momentum network moment.



CLDA: Contrastive Learning for Semi-Supervised Domain Adaptation (Supplementary Material)

Neural Information Processing Systems

The supplementary material consists of the following. Additional Results of the DomainNet dataset for 5 and 10-shot settings with Resnet34 as backbone network are shown in Table 1. Results are reported in Tables 2 and 3 Discussion on Limitations and Societal Impacts. The architecture of the network is similar to [2]. All other hyperparameters used in our framework are described in the main paper.



Appendix Implementation Details

Neural Information Processing Systems

A.1 Network Architectures We adopt Daformer [17] with Swin-B or MiT-B5 backbone as the base semantic segmentation architecture. For the segmentation head, we utilize the same head as Daformer [17]. The stem module contains one fully-convolutional layers with kernel 3 3 and stride of 2, two fully-convolutional layers with kernel 3 3 and stride of 1, two fully-convolutional layers with kernel 3 3 and stride of 2, and another three fully-convolutional layers with kernel 1 1 and stride of 1 to adjust channels of different feature maps. Level embedding module is defined as metrics with shape 3 dims. The prompt Interactor module contains three fully-convolutional layers with kernel 3 3 and stride of 2 to adjust feature dimensions.