This has resulted in t.he building of systems with semanti(cid:173) cally nameable nodes which perform inferencing by examining t.he pat,t.erns of activation spread.

In this paper we describe the VLSI design and testing of a high capacity associative memory which we call the exponential cor(cid:173) relation associative memory (ECAM). The prototype 3J.'-CMOS programmable chip is capable of storing 32 memory patterns of 24 bits each. The high capacity of the ECAM is partly due to the use of special exponentiation neurons, which are implemented via sub-threshold MOS transistors in this design. The prototype chip is capable of performing one associative recall in 3 J.'S.

A generic model of oscillating cortex, which assumes "minimal" coupling justified by known anatomy, is shown to function as an as(cid:173) sociative memory, using previously developed theory. The network has explicit excitatory neurons with local inhibitory interneuron feedback that forms a set of nonlinear oscillators coupled only by long range excitatofy connections. Using a local Hebb-like learning rule for primary and higher order synapses at the ends of the long range connections, the system learns to store the kinds of oscil(cid:173) lation amplitude patterns observed in olfactory and visual cortex. This rule is derived from a more general "projection algorithm" for recurrent analog networks, that analytically guarantees content addressable memory storage of continuous periodic sequences - capacity: N /2 Fourier components for an N node network - "spurious" attractors.

The CHAC storage scheme has been used as a basis for a software implementation of an associative .emory A major this CHAC-concept is that the disadvantage of degree of local generalization (area of interpo(cid:173) lation) is fixed. This paper deals with an algo(cid:173) rithm for self-organizing variable generaliza(cid:173) tion for the AKS, based on ideas of T. Kohonen.

Combining neuropharmacological experiments with computational model(cid:173) ing, we have shown that cholinergic modulation may enhance associative memory function in piriform (olfactory) cortex. We have shown that the acetylcholine analogue carbachol selectively suppresses synaptic transmis(cid:173) sion between cells within piriform cortex, while leaving input connections unaffected. When tested in a computational model of piriform cortex, this selective suppression, applied during learning, enhances associative memory performance.

The Hopfield network (Hopfield, 1982,1984) provides a simple model of an associative memory in a neuronal structure. This model, however, is based on highly artificial assumptions, especially the use of formal-two state neu(cid:173) rons (Hopfield, 1982) or graded-response neurons (Hopfield, 1984). First, we show that a simple model of a neuron can capture all relevant features of neuron spiking, i. e., a wide range of spiking frequencies and a realistic distribution of interspike inter(cid:173) vals. Second, we construct an associative memory by linking these neurons together. The analytical solution for a large and fully connected network shows that the Hopfield solution is valid only for neurons with a short re(cid:173) fractory period.

A model of the hippocampus is presented which forms rapid self -orga(cid:173) nized representations of input arriving via the perforant path, performs recall of previous associations in region CA3, and performs comparison of this recall with afferent input in region CA 1. This comparison drives feedback regulation of cholinergic modulation to set appropriate dynamics for learning of new representations in region CA3 and CA 1. The network responds to novel patterns with increased cholinergic mod(cid:173) ulation, allowing storage of new self-organized representations, but responds to familiar patterns with a decrease in acetylcholine, allowing recall based on previous representations. This requires selectivity of the cholinergic suppression of synaptic transmission in stratum radiatum of regions CA3 and CAl, which has been demonstrated experimentally.

Selective suppression of transmission at feedback synapses during learning is proposed as a mechanism for combining associative feed(cid:173) back with self-organization of feed forward synapses. Experimental data demonstrates cholinergic suppression of synaptic transmission in layer I (feedback synapses), and a lack of suppression in layer IV (feed(cid:173) forward synapses). A network with this feature uses local rules to learn mappings which are not linearly separable. During learning, sensory stimuli and desired response are simultaneously presented as input. Feedforward connections form self-organized representations of input, while suppressed feedback connections learn the transpose of feedfor(cid:173) ward connectivity.

Similarity based fault tolerant retrieval in neural associative mem(cid:173) ories (N AM) has not lead to wiedespread applications. A draw(cid:173) back of the efficient Willshaw model for sparse patterns [Ste61, WBLH69], is that the high asymptotic information capacity is of little practical use because of high cross talk noise arising in the retrieval for finite sizes. Here a new bidirectional iterative retrieval method for the Willshaw model is presented, called crosswise bidi(cid:173) rectional (CB) retrieval, providing enhanced performance. We dis(cid:173) cuss its asymptotic capacity limit, analyze the first step, and com(cid:173) pare it in experiments with the Willshaw model. Applying the very efficient CB memory model either in information retrieval systems or as a functional model for reciprocal cortico-cortical pathways requires more than robustness against random noise in the input: Our experiments show also the segmentation ability of CB-retrieval with addresses containing the superposition of pattens, provided even at high memory load.

Motivated by the findings of modular structure in the association cortex, we study a multi-modular model of associative memory that can successfully store memory patterns with different levels of ac(cid:173) tivity. We show that the segregation of synaptic conductances into intra-modular linear and inter-modular nonlinear ones considerably enhances the network's memory retrieval performance. Compared with the conventional, single-module associative memory network, the multi-modular network has two main advantages: It is less sus(cid:173) ceptible to damage to columnar input, and its response is consistent with the cognitive data pertaining to category specific impairment.