Score function-based natural language generation (NLG) approaches such as REINFORCE, in general, suffer from low sample efficiency and training instability problems. This is mainly due to the non-differentiable nature of the discrete space sampling and thus these methods have to treat the discriminator as a black box and ignore the gradient information. To improve the sample efficiency and reduce the variance of REINFORCE, we propose a novel approach, TaylorGAN, which augments the gradient estimation by off-policy update and the first-order Taylor expansion. This approach enables us to train NLG models from scratch with smaller batch size --- without maximum likelihood pre-training, and outperforms existing GAN-based methods on multiple metrics of quality and diversity.

In equation (11), will the importance sampling of y also cause high-variance during the estimation? Is this also affected by the accuracy of neighborhood distribution (joint distribution) in equation 13? If yes, what's the performance if you use other types of kernel functions? Why don't use pre-trained language models to evaluate the PPL? e.g., GPT-2 Other comments: 1. It is helpful to elaborate more or mention the joint distribution will be introduced in 3.1.2,

This paper proposed a novel method for GAN-based natural language generation, where first order Taylor expension is used to estimate the gradient of the reword function. This method greatly mitigate the high variance problem of previous methods and improve the sample efficiency. Experiments show the proposed method achieve the state-of-the-art. The work is solid both in theory and in experiments.

Score function-based natural language generation (NLG) approaches such as REINFORCE, in general, suffer from low sample efficiency and training instability problems. This is mainly due to the non-differentiable nature of the discrete space sampling and thus these methods have to treat the discriminator as a black box and ignore the gradient information. To improve the sample efficiency and reduce the variance of REINFORCE, we propose a novel approach, TaylorGAN, which augments the gradient estimation by off-policy update and the first-order Taylor expansion. This approach enables us to train NLG models from scratch with smaller batch size --- without maximum likelihood pre-training, and outperforms existing GAN-based methods on multiple metrics of quality and diversity.