A Additional Training and Architecture Details

A.1 Training details We utilize the Adam optimizer with a learning rate of 0.0001, β Additionally, we implement learning rate warm-up for the initial 1,000 iterations. The minimum number K of objects in each scene is customized separately as follows: K = 8, 7, 5, 5, 2, and 5 for the CLEVR-567, CLEVR-3D, Room-Chair, Room-Diverse, LLFF, and MSN datasets, respectively. To allow training on a high resolution, such as 256 256, we render individual pixels instead of large-sized patches. Specifically, we randomly sample a batch of 64 rays from the set of all pixels in the dataset, and then follow the hierarchical volume sampling to query 64 samples from the coarse network and 128 samples from the fine network. In addition, we train our model from scratch, with the exception of the Room-Diverse dataset.