Reviews: Are GANs Created Equal? A Large-Scale Study

This paper introduces a large set of experiments to compare recently proposed GANs. It discusses two previously proposed measures -- inception score (IS) and Frechet Inception Distance (FID); and it proposes a new measure in the context of GAN assessment, based on precision, recall and F1. Precision (P) is measured as the fraction of generated samples with distance below a pre-defined threshold \delta; while recall (R) is measured as the fraction of inversely generated samples (from test set) with squared Euclidean distance below \delta (F1 is the usual mean between P and R). The paper argues that IS only measures precision and FIS measures both, so IS is essentially dropped as a measurement for GANs. Then the paper argues that it is important to show the mean and variance of FID and P-R-F1 measurements instead of the best values, computed over a set of random initialisations and hyper-parameter search points.