Embodied Visuomotor Representation

You don't know the precise distance from your eye to any particular object in meters. However, you can immediately reach out and touch any of them. Instead of the meter, your knowledge of distance is encoded in unknown but embodied units of action. In contrast, standard approaches in robotics assume calibration to the meter, so that separated vision and control processes can be interfaced. Consequently, robots are precisely manufactured and calibrated, resulting in expensive systems available in only a few configurations. In response, we propose Embodied Visuomotor Representation, a framework that allows distance to be measured by a robot's own actions and thus minimizes dependence on calibrated 3D sensors and physical models. Using it, we demonstrate that a robot without knowledge of its size, environmental scale, or its own strength can become capable of touching and clearing obstacles after several seconds of operation. Similarly, we demonstrate in simulation that an agent, without knowledge of its mass or strength, can jump a gap of unknown size after performing a few test oscillations. This allows vision and low-level control to be abstracted by the implicit assumption of an external scale, such as the meter, to coordinate them. For example, it is common to construct a passive visual process that estimates distances and builds a metric map scaled to the meter. Next, the geometry of the world is used by a planning algorithm to design a trajectory scaled to the meter. Then a pre-tuned low-level controller uses feedback to follow the metric trajectory by mapping it to motor signals. This is called the sense, plan, act paradigm, and it has its roots in the Marr paradigm of vision (1). Figure 1 shows a block diagram of the sense-plan-act paradigm. The sense-plan-act architecture allows largely separate teams of engineers and scientists to create equally separate vision and control algorithms tuned for particular tasks and mechanical configurations.