Image-to-Image Translation in Tensorflow - Affine Layer

I thought that the results from pix2pix by Isola et al. looked pretty cool and wanted to implement an adversarial net, so I ported the Torch code to Tensorflow. The single-file implementation is available as pix2pix-tensorflow on github. The network is composed of two main pieces, the Generator and the Discriminator. The Generator applies some transform to the input image to get the output image. The Discriminator compares the input image to an unknown image (either a target image from the dataset or an output image from the generator) and tries to guess if this was produced by the generator.