We sincerely thank our reviewers for the constructive feedback. R4), has strength in visual quality (R1, R3, R4), and performs diverse (R1) and thorough (R3, R4) experiments. Note that we do not claim to be the first to use swapping for disentanglement. We apologize for any confusion. Moreover, our method supports HD resolution (e.g. the mountain example Therefore, we believe our method has advantage over general texture transfer methods.

Neural Information Processing Systems http://nips.cc/

We propose the Swapping Autoencoder, a deep model designed specifically for image manipulation, rather than random sampling. The key idea is to encode an image into two independent components and enforce that any swapped combination maps to a realistic image.

We sincerely thank our reviewers for the constructive feedback. R4), has strength in visual quality (R1, R3, R4), and performs diverse (R1) and thorough (R3, R4) experiments. Note that we do not claim to be the first to use swapping for disentanglement. We apologize for any confusion. Moreover, our method supports HD resolution (e.g. the mountain example Therefore, we believe our method has advantage over general texture transfer methods.

Recently, a team of researchers from UC Berkeley and Adobe Research proposed a new machine learning model known as Swapping Autoencoder, which has the capability to perform image manipulation. The key idea of this research is to encode a picture into 2 independent components and then enforce that any swapped combination maps to a realistic image. Deep generative models such as GANs or Generative Adversarial Networks and Variational Autoencoders (VAEs) have gained much traction by the researchers over the years. According to the researchers, deep generative models have become a popular technique when it comes to producing realistic images from randomly sampled data. However, such deep generative models face various challenges when used for a controllable manipulation of existing images.

We aim to obtain an interpretable, expressive, and disentangled scene representation that contains comprehensive structural and textural information for each object. Previous scene representations learned by neural networks are often uninterpretable, limited to a single object, or lacking 3D knowledge. In this work, we propose 3D scene de-rendering networks (3D-SDN) to address the above issues by integrating disentangled representations for semantics, geometry, and appearance into a deep generative model. Our scene encoder performs inverse graphics, translating a scene into a structured object-wise representation. Our decoder has two components: a differentiable shape renderer and a neural texture generator. The disentanglement of semantics, geometry, and appearance supports 3D-aware scene manipulation, e.g., rotating and moving objects freely while keeping the consistent shape and texture, and changing the object appearance without affecting its shape. Experiments demonstrate that our editing scheme based on 3D-SDN is superior to its 2D counterpart.

We aim to obtain an interpretable, expressive, and disentangled scene representation that contains comprehensive structural and textural information for each object. Previous scene representations learned by neural networks are often uninterpretable, limited to a single object, or lacking 3D knowledge. In this work, we propose 3D scene de-rendering networks (3D-SDN) to address the above issues by integrating disentangled representations for semantics, geometry, and appearance into a deep generative model. Our scene encoder performs inverse graphics, translating a scene into a structured object-wise representation. Our decoder has two components: a differentiable shape renderer and a neural texture generator. The disentanglement of semantics, geometry, and appearance supports 3D-aware scene manipulation, e.g., rotating and moving objects freely while keeping the consistent shape and texture, and changing the object appearance without affecting its shape. Experiments demonstrate that our editing scheme based on 3D-SDN is superior to its 2D counterpart.