Model Selection for Gaussian Mixture Models

Finite mixture modeling is a flexible and powerful approach to modeling data that is heterogeneous and stems from multiple populations, such as data from patter recognition, computer vision, image analysis, and machine learning. The Gaussian mixture model is an important mixture model family. It is well known that any continuous distribution can be approximated arbitrarily well by a finite mixture of normal densities (Lindsay, 1995; McLachlan and Peel, 2000). However, as demonstrated by Chen (1995), when the number of components is unknown, the optimal convergence rate of the estimate of a finite mixture model is slower than the optimal convergence rate when the number is known. In practice, with too many components, the mixture may overfit the data and yield poor interpretations, while with too few components, the mixture may not be flexible enough to approximate the true underlying data structure.