Paper Review: A Deep Dive into Imagen

Investigating the first half of this claim, the authors present several qualitative comparisons between Imagen and DALL-E 2 generated images. They also provide results from human evaluation experiments where people were asked to choose the most photorealistic image from a single text prompt or caption. Even before considering any results, immediately the authors have introduced a degree of subjectivity into their analysis that is inherent in human evaluation experiments. Therefore the results shown in [1] must be considered with care and a healthy level of skepticism. To provide some context to these results, the authors select some example comparisons shown to human raters and include these in the Appendix (definitely take a look at these -- for motivation, I've added an example from DALL-E 2 above). However, even with these examples, I find it difficult to make a clear judgement over which image should be preferred. Considering the copied examples shown in the figure above, personally I believe that some of DALL-E 2's generated images are more photorealistic than Imagen's, which demonstrates the issues of subjectivity when collecting results such as these. The authors choose to ask human raters'which image is more photorealistic?'