Learning Spatially Structured Image Transformations Using Planar Neural Networks

Learning Spatially Structured Image Transformations Using Planar Neural Networks Joel Michelson, Joshua H. Palmer, Aneesha Dasari, and Maithilee Kunda Electrical Engineering and Computer Science, V anderbilt University, Nashville TN, USA Abstract --Learning image transformations is essential to the idea of mental simulation as a method of cognitive inference. We take a connectionist modeling approach, using planar neural networks to learn fundamental imagery transformations, like translation, rotation, and scaling, from perceptual experiences in the form of image sequences. We investigate how variations in network topology, training data, and image shape, among other factors, affect the efficiency and effectiveness of learning visual imagery transformations, including effectiveness of transfer to operating on new types of data. I NTRODUCTION Visuospatial reasoning is ubiquitous in everyday human intelligence. In addition to its reliance on semantic knowledge about objects, categories, and scenes, visuospatial reasoning also requires non-semantic knowledge about object shapes, spatial relationships, etc., including, for example [1] (p. 182): "Transforming the spatial codings of objects, including expansions or reductions in size, rotation, [etc.]...accumulating sequences of such changes and visualizing change over time...." We do not know exactly how the human brain represents such non-semantic visuospatial knowledge about transformations, but we do know that this knowledge is learned through real-world perceptual experiences, especially in infancy and early childhood [2]; and that it is often deployed through top-down neural activations in brain regions associated with visual perception, i.e., using visual mental imagery [3]. Only a few studies have examined how AI systems can represent and learn transformation-based reasoning operations like image rotation from perceptual experience. One early study represented each operation as a distributed set of weights in a single-layer, 2D connectionist network, and used the perceptron learning rule to learn each operation in a supervised fashion from image sequences depicting that operation [4].