Reviews: Paraphrase Generation with Latent Bag of Words

Thus paper presents a model where a latent bag-of-words inform a paraphrase generation model. For each source words, the authors compute a multinomial over "neighbor" vocabulary words; this then yields a bag-of-words by a mixture of softmaxes over these neighbors. In the generative process, a set of words is drawn from this distribution, then their word embeddings are averaged to form input to the decoder. During training, the authors use a continuous relaxation of this with Gumbel top-k sampling (a differentiable way to sample k of these words without replacement). The words are averaged and fed into the LSTM's initial state.