Also, forsimplicity, assume t =1 .

Unsupervised text embedding has shown great power in a wide range of NLP tasks. While text embeddings are typically learned in the Euclidean space, directional similarity is often more effective in tasks such as word similarity and document clustering, which creates a gap between the training stage and usage stage of text embedding. To close this gap, we propose a spherical generative model based on which unsupervised word and paragraph embeddings are jointly learned. To learn text embeddings in the spherical space, we develop an efficient optimization algorithm with convergence guarantee based on Riemannian optimization. Our model enjoys high efficiency and achieves state-of-the-art performances on various text embedding tasks including word similarity and document clustering.

This paper proposes JoSE, a method to train word embeddings. Their unsupervised approach is rooted in the principle that words with similar contexts should be similar, where they have some novelty in their generative model using both word-word and word-paragraph embeddings and the novelty largely lies in their constraint that all embeddings are on the unit sphere - where they derive an optimization procedure for this constrained problem using Riemannian optimization. They also utilize word, paragraph s The empirical results form this paper are strong - outperforming the GloVe, Poincare Glove, and Word2vec baselines considerably in some cases. FastText is also outperformed as well, though less so, but FastText does have the advantage of using character n-gram information which is not used in JoSE. They also evaluate on analogies and embedding documents from the 20 newsgroups dataset and clustering them, evaluating on the purity of the clusters.

Unsupervised text embedding has shown great power in a wide range of NLP tasks. While text embeddings are typically learned in the Euclidean space, directional similarity is often more effective in tasks such as word similarity and document clustering, which creates a gap between the training stage and usage stage of text embedding. To close this gap, we propose a spherical generative model based on which unsupervised word and paragraph embeddings are jointly learned. To learn text embeddings in the spherical space, we develop an efficient optimization algorithm with convergence guarantee based on Riemannian optimization. Our model enjoys high efficiency and achieves state-of-the-art performances on various text embedding tasks including word similarity and document clustering.

Unsupervised text embedding has shown great power in a wide range of NLP tasks. While text embeddings are typically learned in the Euclidean space, directional similarity is often more effective in tasks such as word similarity and document clustering, which creates a gap between the training stage and usage stage of text embedding. To close this gap, we propose a spherical generative model based on which unsupervised word and paragraph embeddings are jointly learned. To learn text embeddings in the spherical space, we develop an efficient optimization algorithm with convergence guarantee based on Riemannian optimization. Our model enjoys high efficiency and achieves state-of-the-art performances on various text embedding tasks including word similarity and document clustering.