HandCept: A Visual-Inertial Fusion Framework for Accurate Proprioception in Dexterous Hands

--As robotics progresses toward general manipulation, dexterous hands are becoming increasingly critical. However, proprioception in dexterous hands remains a bottleneck due to limitations in volume and generality. In this work, we present HandCept, a novel visual-inertial proprioception framework designed to overcome the challenges of traditional joint angle estimation methods. HandCept addresses the difficulty of achieving accurate and robust joint angle estimation in dynamic environments where both visual and inertial measurements are prone to noise and drift. T o support sim-to-real transfer, we also open-sourced our high-fidelity rendering pipeline, which is essential for training without real-world ground truth. This work offers a robust, generalizable solution for proprioception in dexterous hands, with significant implications for robotic manipulation and human-robot interaction. Performing these tasks often involves manipulation--either bimanual or in-hand dexterous manipulation--to interact with the environment. The end-effector plays a fundamental role in enabling such robotic manipulation [3]-[5].