Factorial Learning and the EM Algorithm

Many real world learning problems are best characterized by an interaction of multiple independent causes or factors. Discovering suchcausal structure from the data is the focus of this paper. Based on Zemel and Hinton's cooperative vector quantizer (CVQ) architecture, an unsupervised learning algorithm is derived from the Expectation-Maximization (EM) framework. Due to the combinatorial natureof the data generation process, the exact E-step is computationally intractable. Two alternative methods for computing theE-step are proposed: Gibbs sampling and mean-field approximation, and some promising empirical results are presented.