ETH Zurich Proposes a Robotic System Capable of Self-Improving Its Semantic Perception

Mobile intelligent robots are being deployed in increasingly unstructured environments, where they are expected to work out complex and dynamic tasks such as autonomous movement and mobile manipulation. Such learning-based robots not only need to acquire basic information about their environments, but must also build this understanding with respect to factors such as object detection and semantic classification. Typically, a static model pretrained on a variety of data is deployed in a particular learning-based robot system. A robot expected to understand semantics, i.e. what is happening in a scene, would therefore learn how to do so during its pretraining phase. This approach poses three main challenges: the model may need to be retrained to incorporate new data; acquired knowledge should be preserved while adapting to new tasks and environments; and training signals of the environment are required during deployment.