[R] Improving WGAN by Allowing Generator to see Discriminator's Hidden States • r/MachineLearning

WGAN has really paved the way for alot of GAN applications both in images and text. However, one problem I primarily see with training WGAN for text is that the generator fails to fully converge. That is, the wasserstein distance still remains large and despite numerous steps, the generator will not converge any further. To aid the generator, one idea is to allow the generator to see the Discriminator's preactivations from hidden layers and allow the generator to revise its outputs. The idea here is that the generator gets a chance to propose a sequence, see how the discriminator will evaluate it, and revise it sequence all in one differentiable calculation.