Accelerated Inference for Latent Variable Models

Bayesian nonparametrics (BNP) models appear to be perfectly suited for the era of big data (Jordan, 2011), in which ever-expanding databases of high-dimensional data cannot be dealt with simplistically. Generative processes priors like the Dirichlet process (Ferguson, 1973) or the Indian buffet process (Griffiths and Ghahramani, 2011) allow for modeling latent variables like clusters or otherwise unobservable features in our data and adapting the complexity of the model in accordance to the complexity of the data. Even if we had some understanding of the latent structure in the data, we would not necessarily know their exact forms and implications in the model a priori. The BNP solution, which divides the data into discrete features and clusters, fosters interpretable models that would naturally lead to new hypotheses about the information in such databases (Kim et al., 2015). For example, in a general medical records dataset containing billions of observations, a cluster (or feature) composed of 0.001% of the population still includes tens of thousands of people.