Color-Pair Guided Robust Zero-Shot 6D Pose Estimation and Tracking of Cluttered Objects on Edge Devices

Abstract-- Robust 6D pose estimation of novel objects under challenging illumination remains a significant challenge, often requiring a trade-off between accurate initial pose estimation and efficient real-time tracking. We present a unified framework explicitly designed for efficient execution on edge devices, which synergizes a robust initial estimation module with a fast motion-based tracker . The key to our approach is a shared, lighting-invariant color-pair feature representation that forms a consistent foundation for both stages. For initial estimation, this feature facilitates robust registration between the live RGB-D view and the object's 3D mesh. Extensive experiments on benchmark datasets demonstrate that our integrated approach is both effective and robust, providing competitive pose estimation accuracy while maintaining high-fidelity tracking even through abrupt pose changes. Estimation of an object's six-degree-of-freedom (6D) pose, which involves determining its 3D rotation and 3D translation relative to a camera, is a fundamental task in computer vision and robotics [1]. Accurate 6D pose information is crucial for a variety of applications, ranging from robotic manipulation and grasping in industrial and household environments to immersive experiences in augmented and mixed reality. The ability of an autonomous system to precisely locate and determine the orientation of objects is a key prerequisite for meaningful physical interaction. Furthermore, in dynamic scenarios, this capability must extend beyond single-frame estimation to continuous, real-time tracking, providing the temporal coherence necessary for tasks such as closed-loop robotic control. Historically, pose estimation has focused on instance-level methods, which require costly, object-specific training and thus cannot generalize to new objects. While category-level approaches can handle unseen instances within a known class, they still fail to address entirely novel categories.