However, the reliance on pairwise prediction and the limited generalization capability inherently restrict the global geometric consistency. In this work, we introduce Test3R, a surprisingly simple test-time learning technique that significantly boosts geometric accuracy. Using image triplets (I1,I2,I3), Test3R generates reconstructions from pairs (I1,I2) and (I1,I3). The core idea is to optimize the network at test time via a self-supervised objective: maximizing the geometric consistency between these two reconstructions relative to the common image I1. This ensures the model produces cross-pair consistent outputs, regardless of the inputs. Extensive experiments demonstrate that our technique significantly outperforms previous state-of-the-art methods on the 3D reconstruction and multiview depth estimation tasks. Moreover, it is universally applicable and nearly cost-free, making it easily applied to other models and implemented with minimal test-time training overhead and parameter footprint. Code is available at https://github.com/nopQAQ/Test3R.

Neural techniques radiance to reconstruct fields (NeRF) and render and 3D photorealistic Gaussian Splatting images. Ho (3DGS) wever, are the popular prerequisite completeness.

Neural radiance fields (NeRF) and 3D Gaussian Splatting (3DGS) are popular techniques to reconstruct and render photorealistic images. However, the prerequisite of running Structure-from-Motion (SfM) to get camera poses limits their completeness. Although previous methods can reconstruct a few unposed images, they are not applicable when images are unordered or densely captured. In this work, we propose a method to train 3DGS from unposed images.

Multi-view transformers such as DUSt3R are revolutionizing 3D vision by solving 3D tasks in a feed-forward manner. However, contrary to previous optimization-based pipelines, the inner mechanisms of multi-view transformers are unclear. Their black-box nature makes further improvements beyond data scaling challenging and complicates usage in safety- and reliability-critical applications. Here, we present an approach for probing and visualizing 3D representations from the residual connections of the multi-view transformers' layers. In this manner, we investigate a variant of the DUSt3R model, shedding light on the development of its latent state across blocks, the role of the individual layers, and suggest how it differs from methods with stronger inductive biases of explicit global pose. Finally, we show that the investigated variant of DUSt3R estimates correspondences that are refined with reconstructed geometry. The code used for the analysis is available at https://github.com/JulienGaubil/und3rstand .

Recovering 4D from monocular video, which jointly estimates dynamic geometry and camera poses, is an inevitably challenging problem. While recent pointmap-based 3D reconstruction methods (e.g., DUSt3R) have made great progress in reconstructing static scenes, directly applying them to dynamic scenes leads to inaccurate results. This discrepancy arises because moving objects violate multi-view geometric constraints, disrupting the reconstruction. To address this, we introduce C4D, a framework that leverages temporal Correspondences to extend existing 3D reconstruction formulation to 4D. Specifically, apart from predicting pointmaps, C4D captures two types of correspondences: short-term optical flow and long-term point tracking. We train a dynamic-aware point tracker that provides additional mobility information, facilitating the estimation of motion masks to separate moving elements from the static background, thus offering more reliable guidance for dynamic scenes. Furthermore, we introduce a set of dynamic scene optimization objectives to recover per-frame 3D geometry and camera parameters. Simultaneously, the correspondences lift 2D trajectories into smooth 3D trajectories, enabling fully integrated 4D reconstruction. Experiments show that our framework achieves complete 4D recovery and demonstrates strong performance across multiple downstream tasks, including depth estimation, camera pose estimation, and point tracking. Project Page: https://littlepure2333.github.io/C4D

RELATED WORKS Hand-Eye Calibration: Hand-eye calibration is a well-established problem in robotics that aims to estimate the relative pose between a camera and a robot's end-effector. It is typically addressed by capturing a series of images of a known calibration pattern (e.g., a checkerboard) using a camera rigidly mounted on the robot hand, and using both the images and the corresponding robot poses to compute the camera's extrinsic parameters. Different mathematical formulations exist for solving hand-eye calibration; a widely adopted approach involves solving the equation AX = XB, where X is the unknown rigid transformation describing the pose of the camera with respect to the robot, while A and B denote the relative motions of the end-effector (from robot kinematics) and the camera (from pattern observations), respectively [31], [36]-[38]. Several other approaches were proposed: Shah [39] formulated a closed-form solution for the hand-eye problem by using an algorithm based on Singular V alue Decomposition (SVD) and the Kronecker product to solve for rotation and translation separately, while Li et al. [40] used dual quaternions to solve them simultaneously overcoming the limitations of the Kronecker product. Wang et al. [23] extended hand-eye calibration to multi-camera setups by incorporating a common reference frame but required an external motion capture system, limiting its applicability to small setups. Andreff and Heller [41], [42] proposed two similar hand-eye calibration methods that leverage the Structure-from-Motion (SfM) paradigm to estimate camera motion and introduce a formulation for hand-eye calibration that includes a factor to metrically scale camera poses.

Hand-object 3D reconstruction has become increasingly important for applications in human-robot interaction and immersive AR/VR experiences. A common approach for object-agnostic hand-object reconstruction from RGB sequences involves a two-stage pipeline: hand-object 3D tracking followed by multi-view 3D reconstruction. However, existing methods rely on keypoint detection techniques, such as Structure from Motion (SfM) and hand-keypoint optimization, which struggle with diverse object geometries, weak textures, and mutual hand-object occlusions, limiting scalability and generalization. As a key enabler to generic and seamless, non-intrusive applicability, we propose in this work a robust, keypoint detector-free approach to estimating hand-object 3D transformations from monocular motion video/images. W e further integrate this with a multi-view reconstruction pipeline to accurately recover hand-object 3D shape. Our method, named HOSt3R, is unconstrained, does not rely on pre-scanned object templates or camera intrinsics, and reaches state-of-the-art performance for the tasks of object-agnostic hand-object 3D transformation and shape estimation on the SHOWMe benchmark. W e also experiment on sequences from the HO3D dataset, demonstrating generalization to unseen object categories.

Understanding and predicting the dynamics of the physical world can enhance a robot's ability to plan and interact effectively in complex environments. While recent video generation models have shown strong potential in modeling dynamic scenes, generating videos that are both temporally coherent and geometrically consistent across camera views remains a significant challenge. To address this, we propose a 4D video generation model that enforces multi-view 3D consistency of videos by supervising the model with cross-view pointmap alignment during training. This geometric supervision enables the model to learn a shared 3D representation of the scene, allowing it to predict future video sequences from novel viewpoints based solely on the given RGB-D observations, without requiring camera poses as inputs. Compared to existing baselines, our method produces more visually stable and spatially aligned predictions across multiple simulated and real-world robotic datasets. We further show that the predicted 4D videos can be used to recover robot end-effector trajectories using an off-the-shelf 6DoF pose tracker, supporting robust robot manipulation and generalization to novel camera viewpoints.

Room layout estimation from multiple-perspective images is poorly investigated due to the complexities that emerge from multi-view geometry, which requires muti-step solutions such as camera intrinsic and extrinsic estimation, image matching, and triangulation. However, in 3D reconstruction, the advancement of recent 3D foundation models such as DUSt3R has shifted the paradigm from the traditional multi-step structure-from-motion process to an end-to-end single-step approach. To this end, we introduce Plane-DUSt3R, a novel method for multi-view room layout estimation leveraging the 3D foundation model DUSt3R. Plane-DUSt3R incorporates the DUSt3R framework and fine-tunes on a room layout dataset (Structure3D) with a modified objective to estimate structural planes. By generating uniform and parsimonious results, Plane-DUSt3R enables room layout estimation with only a single post-processing step and 2D detection results. Unlike previous methods that rely on single-perspective or panorama image, Plane-DUSt3R extends the setting to handle multiple-perspective images. Moreover, it offers a streamlined, end-to-end solution that simplifies the process and reduces error accumulation. Experimental results demonstrate that Plane-DUSt3R not only outperforms state-of-the-art methods on the synthetic dataset but also proves robust and effective on in the wild data with different image styles such as cartoon. Our code is available at: https://github.com/justacar/Plane-DUSt3R

To perform Structure-from-Motion (SfM) is the task of jointly recovering SfM from an image collection, DUSt3R works [22, camera poses and reconstructing the 3D scene 51] first compute stereo reconstruction exhaustively for all structure from a set of unconstrained images. This longstanding image pairs and then obtain globally aligned pointmaps problem is essential to many computer vision applications, for all cameras through joint optimization of pairwise rigid including novel view synthesis via NeRFs [3, transformations and local pointmaps. This baseline has been 29] and 3DGS [20], multi-view stereo (MVS) reconstruction significantly improved by the concurrent work MASt3R- [31, 49], and visual localization [34, 36]. Traditional SfM [12] that leverages image retrieval to drastically reduce SfM methods generally follow two main approaches: incremental the computation overhead, boosts optimization efficiency [37, 41, 56] and global [8, 30, 55] SfM. Both by optimizing only over the sparse pixel correspondences, paradigms rely on key components such as feature detection and appends a global bundle adjustment stage for and matching for correspondence search, 3D triangulation accuracy refinement. While optimization-based alignment to reconstruct geometry from 2D correspondences, has been proven to be the key to accurate 3D reconstruction and joint optimization of camera poses and scene geometry by DUSt3R, MASt3R-SfM and classical SfM methods through bundle adjustment. A major research direction has [25, 30, 37], this comes at the cost of slow runtime and been to replace these components with learning-based modules, extensive memory footprint even for moderately-sized image progressing towards fully end-to-end SfM [7, 40, 50].