Foveation in the Era of Deep Learning

Many biological vision systems sense the world with a foveated sensor, where the highest resolution processing is limited to only a small central portion of the visual field (the fovea). Computer vision systems have taken inspiration from this aspect of biological vision and incorporated it into visual attention models that learn to sample and process visual scenes actively [1, 2, 3]. The promise of foveated vision is the ability to resolve and process fine details while simultaneously maintaining a wide field of view, which has applications to problems where semantic information can exist over a high-dynamic range of scales. More generally, it is well known that scaling the resolution of inputs to CNNs can reliably improve accuracy in objection recognition problems [4]. Through sparse sampling in the periphery of the field of view, foveated sensors can achieve this with significantly fewer pixels than a uniform sensor, making it an appealing approach to building parsimonious vision systems.