Optimal Sampling of Natural Images: A Design Principle for the Visual System

One ofthe major theoretical issues in neural computation is to understand how this efficiency is reached given the constraints imposed by the biological hardware. Part of the problem [2] is simply to give an informative representation ofthe visual world using a limited number of neurons, each of which has a limited information capacity. The information capacity of the visual system is determined in part by the spatial transfer characteristics, or "receptive fields," of the individual cells. From a theoretical point of view we can ask if there exists an optimal choice for these receptive fields, a choice which maximizes the information transfer through the system given the hardware constraints. We show that this optimization problem has a simple formulation which allows us to use the intuition developed through the variational approach to quantum mechanics. In general our approach leads to receptive fields which are quite unlike those observed forcells in the visual cortex. In particular orientation selectivity is not a generic prediction. The optimal filters, however, depend on the statistical properties ofthe images we are trying to sample. Natural images have a symmetry - scale invariance [4] - which saves the theory: The optimal receptive fields for sampling of natural images are indeed orientation selective and bear a striking resemblance to observed receptive field characteristics in the mammalian visual cortex as well as the retinal ganglion of lower vertebrates.