Variational Inference for Bayesian Mixtures of Factor Analysers

We present an algorithm that infers the model structure of a mix(cid:173) ture of factor analysers using an efficient and deterministic varia(cid:173) tional approximation to full Bayesian integration over model pa(cid:173) rameters. This procedure can automatically determine the opti(cid:173) mal number of components and the local dimensionality of each component (Le. the number of factors in each factor analyser) . Alternatively it can be used to infer posterior distributions over number of components and dimensionalities. Since all parameters are integrated out the method is not prone to overfitting. Using a stochastic procedure for adding components it is possible to per(cid:173) form the variational optimisation incrementally and to avoid local maxima.