An extremely compact, all analog and fully parallel implementa(cid:173) tion of a class of shunting recurrent neural networks that is ap(cid:173) plicable to a wide variety of FET-based integration technologies is proposed. While the contrast enhancement, data compression, and adaptation to mean input intensity capabilities of the network are well suited for processing of sensory information or feature extrac(cid:173) tion for a content addressable memory (CAM) system, the network also admits a global Liapunov function and can thus achieve stable CAM storage itself. In addition the model can readily function as a front-end processor to an analog adaptive resonance circuit.

We describe a stochastic, dynamical system capable of inference and learning in a probabilistic latent variable model. The most challenging problem in such models - sampling the posterior distribution over latent variables - is proposed to be solved by harnessing natural sources of stochasticity inherent in electronic and neural systems. We demonstrate this idea for a sparse coding model by deriving a continuous-time equation for inferring its latent variables via Langevin dynamics. The model parameters are learned by simultaneously evolving according to another continuous-time equation, thus bypassing the need for digital accumulators or a global clock. Moreover we show that Langevin dynamics lead to an efficient procedure for sampling from the posterior distribution in the 'L0 sparse' regime, where latent variables are encouraged to be set to zero as opposed to having a small L1 norm. This allows the model to properly incorporate the notion of sparsity rather than having to resort to a relaxed version of sparsity to make optimization tractable. Simulations of the proposed dynamical system on both synthetic and natural image datasets demonstrate that the model is capable of probabilistically correct inference, enabling learning of the dictionary as well as parameters of the prior.

Spiking neural networks (SNNs) often are touted as a way to get close to the power efficiency of the brain, but there is widespread confusion about what exactly that means. In fact, there is disagreement about how the brain actually works. Some SNN implementations are less brain-like than others. Depending on whom you talk to, SNNs are either a long way away or close to commercialization. The varying definitions of SNNs leads to differences in how the industry is seen. "A few startups are doing their own SNNs," said Ron Lowman, strategic marketing manager of IP at Synopsys. "It's being driven by guys that have expertise in how to train, optimize, and write software for them." On the other hand, Flex Logix Inference Technical Marketing Manager Vinay Mehta said that, "SNNs are out further than reinforcement learning," referring to a machine-learning concept that's still largely in the research phase. The entire notion of a "neural network" is motivated by attempts to model how the brain works.

The first case shows recurrent activity, while the second case is non-recurrent or feed forward. The polarity of these terms signify excitatory or inhibitory interactions. Shunting network equations can be derived from various sources such as the passive membrane equation with synaptic interaction (Grossberg 1973, Pinter 1983), models of dendritic interaction (RaIl 1977), or experiments on motoneurons (Ellias and Grossberg 1975).

The first case shows recurrent activity, while the second case is non-recurrent or feed forward. The polarity of these terms signify excitatory or inhibitory interactions. Shunting network equations can be derived from various sources such as the passive membrane equation with synaptic interaction (Grossberg 1973, Pinter 1983), models of dendritic interaction (RaIl 1977), or experiments on motoneurons (Ellias and Grossberg 1975). While the exact mechanisms of synaptic interactions are not known in every individual case,neurobiological evidence of shunting interactions appear in several 696 Nabet, Darling and Pinter areas such as sensory systems, cerebellum, neocortex, and hippocampus (Grossberg 1973, Pinter 1987). In addition to neurobiology, these networks have been used to successfully explain data from disciplines ranging from population biology (Lotka 1956) to psychophysics and behavioral psychology (Grossberg 1983). Shunting nets have important advantages over additive models which lack the extra nonlinearityintroduced by the multiplicative terms.