Teaching Perception

T eaching Perception Jonathan H. Connell 1 Abstract -- The visual world is very rich and generally too complex to perceive in its entirety. Y et only certain features are typically required to adequately perform some task in a given situation. Rather than hardwire-in decisions about when and what to sense, this paper describes a robotic system whose behavioral policy can be set by verbal instructions it receives. These capabilities are demonstrated in an associated video [1] showing the fully implemented system guiding the perception of a physical robot in simple scenario. The structure and functioning of the underlying natural language based symbolic reasoning system is also discussed. I. INTRODUCTION Sensing is not without costs. For any given object there are many things that can be known about it. What constitutes a reasonable amount of information to obtain? For instance, to identify an object in a scene a robot could run a DNN recognizer. But, depending on the resources available, this may take a noticeable amount of time. And, while some recognizers have Nary outputs, others are designed as one-versus-all. In this case, to classify an object a robot might have to run N separate nets.