Learning Hierarchical Sparse Representations using Iterative Dictionary Learning and Dimension Reduction

Working towards a Computational Theory of Intelligence, we develop a computational framework inspired by ideas from Neuroscience. Specifically, we integrate notions of columnar organization, hierarchical structure, sparse distributed representations, and sparse coding. An integrated view of Intelligence has been proptosed by Karl Friston based on free-energy [13, 11, 8, 9, 10, 12]. In this framework, Intelligence is viewed as a surrogate minimization of the entropy of this sensorium. This work is intuitively inspired by this view, aiming to provide a computational foundation for a theory of intelligence from the perspective of theoretical computer science, thereby connecting to ideas in mathematics. By building foundations for a principled approach, the computational essence of problems can be isolated, formalized, and their relationship to fundamental problems in mathematics and theoretical computer science can be illuminated and the full power of available mathematical techniques can be brought to bear. A computational approach is focused on developing tractable algorithms.