Dual Attention GANs for Semantic Image Synthesis

In this paper, we focus on the semantic image synthesis task that aims at transferring semantic label maps to photo-realistic images. Existing methods lack effective semantic constraints to preserve the semantic information and ignore the structural correlations in both spatial and channel dimensions, leading to unsatisfactory blurry and artifact-prone results. To address these limitations, we propose a novel Dual Attention GAN (DAGAN) to synthesize photo-realistic and semantically-consistent images with fine details from the input layouts without imposing extra training overhead or modifying the network architectures of existing methods. We also propose two novel modules, i.e., position-wise Spatial Attention Module Figure 1: Visualization of generated semantic maps compared (SAM) and scale-wise Channel Attention Module (CAM), to capture with those from GauGAN [31] on Cityscapes (top) and semantic structure attention in spatial and channel dimensions, ADE20K (bottom). Equipped with semantic attention modeling respectively. Specifically, SAM selectively correlates the pixels at in both spatial and channel dimensions, the proposed each position by a spatial attention map, leading to pixels with the DAGAN can achieve mutual gains within the regions with same semantic label being related to each other regardless of their the same semantic label regardless of the distances, thus improving spatial distances. Meanwhile, CAM selectively emphasizes the scalewise intra-class semantic consistency. Most improved regions features at each channel by a channel attention map, which are highlighted in the ground truths with white dash integrates associated features among all channel maps regardless of boxes.