We introduce Stanford-ORB, a new real-world 3DObject inverse Rendering Benchmark. Recent advances in inverse rendering have enabled a wide range of real-world applications in 3D content generation, moving rapidly from research and commercial use cases to consumer devices. While the results continue to improve, there is no real-world benchmark that can quantitatively assess and compare the performance of various inverse rendering methods. Existing real-world datasets typically consist only of the shape and multi-view images of objects, which are not sufficient for evaluating the quality of material recovery and object relighting. Methods capable of recovering material and lighting often resort to synthetic data for quantitative evaluation, which on the other hand does not guarantee generalization to complex real-world environments. We introduce a new dataset of real-world objects captured under a variety of natural scenes with ground-truth 3D scans, multi-view images, and environment lighting. Using this dataset, we establish the first comprehensive real-world evaluation benchmark for object inverse rendering tasks from in-thewild scenes and compare the performance of various existing methods. All data, code, and models can be accessed at https://stanfordorb.github.io/.

Our main reconstruction loss is an MSE between the rendered color c and the corresponding pixel in the input image. This loss is then exponentially faded over 100,000 steps to a cosine weighted MSE: (x ωo n ˆxωo n)2. This weighting tends to achieve better BRDF fitting results [4] as harsh grazing highlights from the Fresnel effect are not factored as much as regular samples, as well as our approximated rendering model being the least accurate in the grazing angles. The reason for this fading loss scheme is that the normals nare not reliable in the early stages of the training.

Severalworksextend thesetodynamic scenes captured with monocular video, with promising performance.

While 3D Gaussians efficiently approximate an object's surface, they fail to capture the volumetric properties of subsurface scattering. We propose a framework for optimizing an object's shape together with the radiance transfer field given multi-view OLA T (one light at a time) data.

However error control for inverse graphics is difficult, and inverse graphics methods can represent only the effects of the chosen intrinsics.

We introduce Stanford-ORB, a new real-world 3D Object inverse Rendering Benchmark.