GAN You Hear Me? Reclaiming Unconditional Speech Synthesis from Diffusion Models