Correlative Information Maximization Based Biologically Plausible Neural Networks for Correlated Source Separation

The brain effortlessly extracts latent causes of stimuli, but how it does this at the network level remains unknown. Most prior attempts at this problem proposed neural networks that implement independent component analysis, which works under the limitation that latent causes are mutually independent. Here, we relax this limitation and propose a biologically plausible neural network that extracts correlated latent sources by exploiting information about their domains. To derive this network, we choose the maximum correlative information transfer from inputs to outputs as the separation objective under the constraint that the output vectors are restricted to the set where the source vectors are assumed to be located. The online formulation of this optimization problem naturally leads to neural networks with local learning rules. Our framework incorporates infinitely many set choices for the source domain and flexibly models complex latent structures. Choices of simplex or polytopic source domains result in networks with piecewise-linear activation functions. We provide numerical examples to demonstrate the superior correlated source separation capability for both synthetic and natural sources. Extraction of latent causes, or sources, of complex stimuli sensed by sensory organs is essential for survival. Due to absence of any supervision in most circumstances, this extraction must be performed in an unsupervised manner, a process which has been named blind source separation (BSS) (Comon & Jutten, 2010; Cichocki et al., 2009).