f4552671f8909587cf485ea990207f3b-Reviews.html

Classic work on associative memories following Hopfield 1982 focused on issues of capacity and performance, usually considering random memories embedded as stable attractors of a dynamical system. Such work usually led to capacities which scale linearly with the size of the network. The present work proposes a neural architecture which is able to reach exponential capacities at the cost of introducing specific low-dimensional structure into the stored patterns. The authors propose a bi-partite architecture of pattern and constraint neurons corresponding to patterns and clusters, and a two-tiered algorithm, based on within and between-module processing aimed at retrieving clean versions of noise corrupted the patterns. The intra-module algorithm operates iteratively based on forward and backward iterations, based on a belief variable.