Super Resolution with SRResnet, SRGAN

While it might be compelling to use the pixel-wise MSE error as a metric to measure the performance of the model and thus resulting in maximizing the PSNR score, this loss definition has some obvious flaws for generating perceptually high-quality images. This is because the MSE based solution is optimized when it outputs the average of all possible solutions, which might be not on the HR image manifold and can be sometimes blurry, and unreal. This phenomena is illustrated in the figure below with the blue patch as the MSE based optimal solution. To solve the problem, the authors first proposed a GAN based solution to capture the natural image manifold, and a hybrid loss of summing the context loss and the adversarial loss. To further improve performance, the authors also came up with an improved context loss, which compares more high level features of the image through looking at intermediate activation of the pre-trained VGG-19 network.