Managing sparsity, time, and quality of inference in topic models

Noname manuscript No. (will be inserted by the editor) Abstract Inference is an integral part of probabilistic topic models, but is often nontrivial to derive an efficient algorithm for a specific model. It is even much more challenging when we want to find a fast inference algorithm which always yields sparse latent representations of documents. In this article, we introduce a simple framework for inference in probabilistic topic models, denoted by FW. This framework is general and flexible enough to be easily adapted to mixture models. It has a linear convergence rate, offers an easy way to incorporate prior knowledge, and provides us an easy way to directly trade off sparsity against quality and time. We demonstrate the goodness and flexibility of FW over existing inference methods by a number of tasks. Finally, we show how inference in topic models with nonconjugate priors can be done efficiently. Keywords Topic modeling · Fast inference · Sparsity · Tradeoff · Greedy sparse approximation 1 Introduction We are interested in the two important problems in developing probabilistic topic models: sparsity and time. The sparsity problem is to infer sparse latent representations of documents, while the second problem asks for an efficient inference algorithm for a topic model. These two problems have been attracting significant interest in recent years, because of their significant impacts and nontrivial nature. Inference is an integral part of any topic models, and is often NPhard (Sontag and Roy, 2011).