Batch and On-Line Parameter Estimation of Gaussian Mixtures Based on the Joint Entropy

In contrast to gradient descentand EM, which estimate the mixture's covariance matrices, the proposed method estimates the inverses of the covariance matrices. Furthennore, the new parameter estimation procedure can be applied in both online and batch settings. We show experimentally that it is typically fasterthan EM, and usually requires about half as many iterations as EM. 1 Introduction Mixture models, in particular mixtures of Gaussians, have been a popular tool for density estimation, clustering, and unsupervised learning with a wide range of applications (see for instance [5, 2] and the references therein). Mixture models are one of the most useful tools for handling incomplete data, in particular hidden variables. For Gaussian mixtures the hidden variables indicate for each data point the index of the Gaussian that generated it.