Neural Computation with Winner-Take-All as the Only Nonlinear Operation

Everybody "knows" that neural networks need more than a single layer of nonlinear units to compute interesting functions. This may be of interest from the point of view of neurophysiology, since only 15% of the synapses in the cortex are inhibitory. In addi(cid:173) tion it is widely believed that there are special microcircuits in the cortex that compute winner-take-all. Complete proofs and further details to these results can be found in [Maass, 2000].