Rethinking conditional GAN training: An approach using geometrically structured latent manifolds