Exclusive Topic Modeling

Two wellknown challenges in topic modeling are: 1)the predominance of the frequently appearing words in the estimated topics; 2) topics are overlapped with common words, making the structure and interpretation difficult. We propose an Exclusive Topic Model (ETM) to tackle these two issues. ETM can identify field-specific keywords and deliver well-structured topics with exclusive words. More specifically, a weighted Lasso penalty is imposed to reduce the predominance of the frequently appearing yet less relevant words automatically and a pairwise Kullback-Leibler divergence penalty is used to implement topics separation. Topic modeling makes use of the word co-occurrence information to estimate topics. Due to the human language habit and structure, certain words appear more frequently than others, e.g. the Zipf's law. Consequently, the frequently appearing words co-occur with more words and thus are predominant in the estimated topics. The phenomenon makes topic interpretation difficult, as general and frequently appearing words take the place of the true exclusive topic words. The semantic coherence of the estimated topics also deteriorates.