Using deep neural networks for accurate hand-tracking on Oculus Quest

Researchers and engineers from Facebook Reality Labs and Oculus have developed what is, as of today, the only fully articulated hand-tracking system for VR that relies entirely on monochrome cameras. The system does not use active depth-sensing technology or any additional equipment (such as instrumented gloves). We will deploy this technology as a software update for Oculus Quest, the cable-free, stand-alone VR headset that is now available to consumers. By using Quest's four cameras in conjunction with new techniques in deep learning and model-based tracking, we achieve a larger interaction volume for hand-tracking than depth-based solutions do, and we do it at a fraction of the size, weight, power, and cost. Processing is done entirely on-device, and the system is optimized to support gestures for interaction, such as pointing and pinch to select.