Training End-to-end Single Image Generators without GANs