Caltech: New Algorithm Helps Autonomous Vehicles Find Themselves, Summer Or Winter

"The rule of thumb is that both images--the one from the satellite and the one from the autonomous vehicle--have to have identical content for current techniques to work. The differences that they can handle are about what can be accomplished with an Instagram filter that changes an image's hues," says Anthony Fragoso (MS '14, PhD '18), lecturer and staff scientist, and lead author of the Science Robotics paper. "In real systems, however, things change drastically based on season because the images no longer contain the same objects and cannot be directly compared." The process--developed by Chung and Fragoso in collaboration with graduate student Connor Lee (BS '17, MS '19) and undergraduate student Austin McCoy--uses what is known as "self-supervised learning." While most computer-vision strategies rely on human annotators who carefully curate large data sets to teach an algorithm how to recognize what it is seeing, this one instead lets the algorithm teach itself.