「sunset at a beach(ビーチの夕日)」や「mountains next to a lake(湖畔の山)」といった短い文章を入力するだけで瞬時にリアルな画像を作るAI「GauGAN2」を、大手半導体メーカー・NVIDIAの研究機関であるNVIDIA Researchが開発しました。GauGAN2のデモサイトも公開されているので、実際に使ってみてわずか数語の説明からどのような画像を生成するのかを確かめてみました。

Nvidia today detailed an AI system called GauGAN2, the successor to its GauGAN model, that lets users create lifelike landscape images that don't exist. Combining techniques like segmentation mapping, inpainting, and text-to-image generation in a single tool, GauGAN2 is designed to create photorealistic art with a mix of words and drawings. "Compared to state-of-the-art models specifically for text-to-image or segmentation map-to-image applications, the neural network behind GauGAN2 produces a greater variety and higher-quality of images," Isha Salian, a member of Nvidia's corporate communications team, wrote in a blog post. "Rather than needing to draw out every element of an imagined scene, users can enter a brief phrase to quickly generate the key features and theme of an image, such as a snow-capped mountain range. This starting point can then be customized with sketches to make a specific mountain taller or add a couple of trees in the foreground, or clouds in the sky."

Nvidia has updated its Canvas real-time painting tool with a new AI model based on GauGAN2 research to achieve a 4x resolution boost. Canvas enables artists to turn simple brushstrokes into realistic landscapes filled with materials including water, grass, snow, mountains, and more. The idea is that concepts can be turned into final versions far quicker than ever before. The free software, which is still in beta, is the perfect example of how AI complements and enhances human abilities rather than replaces. Canvas' latest update achieves close to photorealism with greater definition and fewer artifacts: The software delivers images in up to 1K pixel resolution and the results can be exported to apps like Adobe Photoshop to integrate with an artist's existing workflow. GauGAN2 combines segmentation mapping, inpainting, and text-to-image generation in a single model.

In this example, we present an implementation of the GauGAN architecture proposed in Semantic Image Synthesis with Spatially-Adaptive Normalization. As we proceed with the example, we will discuss each of the different components in further detail. For a thorough review of GauGAN, please refer to this article. We also encourage you to check out the official GauGAN website, which has many creative applications of GauGAN. This example assumes that the reader is already familiar with the fundamental concepts of GANs.

Nvidia's New Canvas tool, which is now available as a beta-free version, is a real-time painting tool GauGAN that can be used by anyone with an NVIDIA RTX GPU. It allows creators to create a rough landscape by painting blobs and then it fills it with convincingly photorealistic content. Each distinct color on the tool represents a different type of feature: water, mountains, grass, ruins, and others. The crude sketch is passed to a generative adversarial network when colors are blobbed onto the canvas. GANs essentially pass content to AI creator that tries to make a realistic image.