Sight Guide: A Wearable Assistive Perception and Navigation System for the Vision Assistance Race in the Cybathlon 2024

--Visually impaired individuals face significant challenges navigating and interacting with unknown situations, particularly in tasks requiring spatial awareness and semantic scene understanding. T o accelerate the development and evaluate the state of technologies that enable visually impaired people to solve these tasks, the Vision Assistance Race (VIS) at the Cybathlon 2024 competition was organized. In this work, we present Sight Guide, a wearable assistive system designed for the VIS. The system processes data from multiple RGB and depth cameras on an embedded computer that guides the user through complex, real-world-inspired tasks using vibration signals and audio commands. Our software architecture integrates classical robotics algorithms with learning-based approaches to enable capabilities such as obstacle avoidance, object detection, optical character recognition, and touchscreen interaction. In a testing environment, Sight Guide achieved a 95.7% task success rate, and further demonstrated its effectiveness during the Cybathlon competition. This work provides detailed insights into the system design, evaluation results, and lessons learned, and outlines directions towards a broader real-world applicability. N 2020, approximately 43 million people worldwide were blind, with an additional 295 million suffering from moderate to severe visual impairments [1]. Despite advancements in medical treatments [2], these numbers are projected to rise by 2050 [1]. For individuals with visual impairments, the lack of visual information about their surroundings poses substantial challenges in daily activities. While infrastructure adaptations, such as making public transport more accessible, can mitigate some difficulties, many everyday tasks remain impracticable for blind individuals. To enhance their autonomy, most visually impaired people rely on assistive technologies. Assistive technologies in this context are hardware-and software-based solutions that help people with disabilities to overcome or to reduce barriers in their lives. Although a variety of vision aids leveraging computer vision and artificial intelligence are available on the market, these solutions are typically limited to specific tasks like text-to-speech conversion [3], description of the surrounding [4], or navigation assistance [5], [6].