An Application of the Principle of Maximum Information Preservation to Linear Systems

I have previously proposed [Linsker, 1987, 1988] a principle of "maximum information preservation," also called the "infomax" principle, that may account for certain aspects of the organization of a layered perceptual network. The principle applies to a layer L of cells (which may be the input layer or an intermediate layer of the network) that provides input to a next layer M. The mapping of the input signal vector L onto an output signal vector M, f:L M, is characterized by a conditional probability density function ("pdf") p(MI L).