Tech designed to aid visually impaired could benefit from human-AI collaboration

Remote sighted assistance (RSA) technology--which connects visually impaired individuals with human agents through a live video call on their smartphones--helps people with low or no vision navigate tasks that require sight. But what happens when existing computer vision technology doesn't fully support an agent in fulfilling certain requests, such as reading instructions on a medicine bottle or recognizing flight information on an airport's digital screen? According to researchers at the Penn State College of Information Sciences and Technology, there are some challenges that cannot be solved with existing computer vision techniques. Instead, the researchers posit that they would be better addressed by humans and AI working together to improve the technology and enhance the experience for both visually impaired users and the agents who support them. In a recent study presented at the 27th International Conference on Intelligent User Interfaces (IUI) in March, the researchers highlighted five emerging problems with RSA that they say warrant new development in human-AI collaboration.