Goto

Collaborating Authors

 up-nerf


UP-NeRF: Unconstrained Pose-Prior-Free Neural Radiance Fields (Supplement)

Neural Information Processing Systems

In this supplementary material, we provide additional implementation details (Appendix A) of our model and visualization of ablation studies (Appendix B) which are not included in our main paper. BARF-W, and BARF-WD are based on [2] because there is no official NeRF-W code available. The detailed architecture of UP-NeRF is shown in the Figure 1. First two authors have an equal contribution. As we mentioned in the main paper, the evaluation process entails two stages, which are test-time pose optimization and appearance optimization.


UP-NeRF: Unconstrained Pose Prior-Free Neural Radiance Field

Neural Information Processing Systems

Neural Radiance Field (NeRF) has enabled novel view synthesis with high fidelity given images and camera poses. Subsequent works even succeeded in eliminating the necessity of pose priors by jointly optimizing NeRF and camera pose. However, these works are limited to relatively simple settings such as photometrically consistent and occluder-free image collections or a sequence of images from a video. So they have difficulty handling unconstrained images with varying illumination and transient occluders.




UP-NeRF: Unconstrained Pose Prior-Free Neural Radiance Field

Neural Information Processing Systems

Neural Radiance Field (NeRF) has enabled novel view synthesis with high fidelity given images and camera poses. Subsequent works even succeeded in eliminating the necessity of pose priors by jointly optimizing NeRF and camera pose. However, these works are limited to relatively simple settings such as photometrically consistent and occluder-free image collections or a sequence of images from a video. So they have difficulty handling unconstrained images with varying illumination and transient occluders. In this paper, we propose UP-NeRF (Unconstrained Pose-prior-free Neural Radiance Fields) to optimize NeRF with unconstrained image collections without camera pose prior.


UP-NeRF: Unconstrained Pose Prior-Free Neural Radiance Field

Neural Information Processing Systems

Neural Radiance Field (NeRF) has enabled novel view synthesis with high fidelity given images and camera poses. Subsequent works even succeeded in eliminating the necessity of pose priors by jointly optimizing NeRF and camera pose. However, these works are limited to relatively simple settings such as photometrically consistent and occluder-free image collections or a sequence of images from a video. So they have difficulty handling unconstrained images with varying illumination and transient occluders. In this paper, we propose UP-NeRF (Unconstrained Pose-prior-free Neural Radiance Fields) to optimize NeRF with unconstrained image collections without camera pose prior.


UP-NeRF: Unconstrained Pose-Prior-Free Neural Radiance Fields

Kim, Injae, Choi, Minhyuk, Kim, Hyunwoo J.

arXiv.org Artificial Intelligence

Neural Radiance Field (NeRF) has enabled novel view synthesis with high fidelity given images and camera poses. Subsequent works even succeeded in eliminating the necessity of pose priors by jointly optimizing NeRF and camera pose. However, these works are limited to relatively simple settings such as photometrically consistent and occluder-free image collections or a sequence of images from a video. So they have difficulty handling unconstrained images with varying illumination and transient occluders. In this paper, we propose $\textbf{UP-NeRF}$ ($\textbf{U}$nconstrained $\textbf{P}$ose-prior-free $\textbf{Ne}$ural $\textbf{R}$adiance $\textbf{F}$ields) to optimize NeRF with unconstrained image collections without camera pose prior. We tackle these challenges with surrogate tasks that optimize color-insensitive feature fields and a separate module for transient occluders to block their influence on pose estimation. In addition, we introduce a candidate head to enable more robust pose estimation and transient-aware depth supervision to minimize the effect of incorrect prior. Our experiments verify the superior performance of our method compared to the baselines including BARF and its variants in a challenging internet photo collection, $\textit{Phototourism}$ dataset.