Collaborating Authors


Guided Disentanglement in Generative Networks Artificial Intelligence

Image-to-image translation (i2i) networks suffer from entanglement effects in presence of physics-related phenomena in target domain (such as occlusions, fog, etc), thus lowering the translation quality and variability. In this paper, we present a comprehensive method for disentangling physics-based traits in the translation, guiding the learning process with neural or physical models. For the latter, we integrate adversarial estimation and genetic algorithms to correctly achieve disentanglement. The results show our approach dramatically increase performances in many challenging scenarios for image translation.

Why Do AI-generated Portraits Fail at Realism?


This is the promise made by AI Gahaku, one of the most popular artificial intelligence engines trained to generate images that resemble old paintings using our own pictures. It is simple as it sounds: You upload a photo of a face, choose the painting style, and voilà, the AI generates an image that looks like a century-old Western portrait ready to be displayed in a museum. The success of AI Gahaku follows the success of many other apps and projects that over the past few years have experimented with user pictures and AI filters. AI Gahaku claims that "various painting styles can be easily applied to it such as Renaissance, Pop Art, Expressionism and many more!" It would be too easy to deem the popularity of AI-generated portraits resembling old paintings as due to the nostalgic enjoyment of figurative art styles that are safer and easier to understand than modern art.

Enhancing Photorealism Enhancement Artificial Intelligence

We present an approach to enhancing the realism of synthetic images. The images are enhanced by a convolutional network that leverages intermediate representations produced by conventional rendering pipelines. The network is trained via a novel adversarial objective, which provides strong supervision at multiple perceptual levels. We analyze scene layout distributions in commonly used datasets and find that they differ in important ways. We hypothesize that this is one of the causes of strong artifacts that can be observed in the results of many prior methods. To address this we propose a new strategy for sampling image patches during training. We also introduce multiple architectural improvements in the deep network modules used for photorealism enhancement. We confirm the benefits of our contributions in controlled experiments and report substantial gains in stability and realism in comparison to recent image-to-image translation methods and a variety of other baselines.

Paint by Word Artificial Intelligence

We investigate the problem of zero-shot semantic image painting. Instead of painting modifications into an image using only concrete colors or a finite set of semantic concepts, we ask how to create semantic paint based on open full-text descriptions: our goal is to be able to point to a location in a synthesized image and apply an arbitrary new concept such as "rustic" or "opulent" or "happy dog." To do this, our method combines a state-of-the art generative model of realistic images with a state-of-the-art text-image semantic similarity network. We find that, to make large changes, it is important to use non-gradient methods to explore latent space, and it is important to relax the computations of the GAN to target changes to a specific region. We conduct user studies to compare our methods to several baselines.

GeoSim: Photorealistic Image Simulation with Geometry-Aware Composition Artificial Intelligence

Scalable sensor simulation is an important yet challenging open problem for safety-critical domains such as self-driving. Current work in image simulation either fail to be photorealistic or do not model the 3D environment and the dynamic objects within, losing high-level control and physical realism. In this paper, we present GeoSim, a geometry-aware image composition process that synthesizes novel urban driving scenes by augmenting existing images with dynamic objects extracted from other scenes and rendered at novel poses. Towards this goal, we first build a diverse bank of 3D objects with both realistic geometry and appearance from sensor data. During simulation, we perform a novel geometry-aware simulation-by-composition procedure which 1) proposes plausible and realistic object placements into a given scene, 2) renders novel views of dynamic objects from the asset bank, and 3) composes and blends the rendered image segments. The resulting synthetic images are photorealistic, traffic-aware, and geometrically consistent, allowing image simulation to scale to complex use cases. We demonstrate two such important applications: long-range realistic video simulation across multiple camera sensors, and synthetic data generation for data augmentation on downstream segmentation tasks.

Approaching Neural Network Uncertainty Realism Machine Learning

Statistical models are inherently uncertain. Quantifying or at least upper-bounding their uncertainties is vital for safety-critical systems such as autonomous vehicles. While standard neural networks do not report this information, several approaches exist to integrate uncertainty estimates into them. Assessing the quality of these uncertainty estimates is not straightforward, as no direct ground truth labels are available. Instead, implicit statistical assessments are required. For regression, we propose to evaluate uncertainty realism -- a strict quality criterion -- with a Mahalanobis distance-based statistical test. An empirical evaluation reveals the need for uncertainty measures that are appropriate to upper-bound heavy-tailed empirical errors. Alongside, we transfer the variational U-Net classification architecture to standard supervised image-to-image tasks. We adopt it to the automotive domain and show that it significantly improves uncertainty realism compared to a plain encoder-decoder model.

AI in the Gaming Industry


Artificial intelligence has been interlinked with the gaming industry since the beginning of video games. As expected, the technology was almost rudimentary at first, but it was there, laying the foundation of what was to be one of the most phenomenal technological revolutions in the world. AI in the gaming industry has completely changed the…rules of the game! Everything started with software-controlled games like the legendary Pac-Man and Pong paddle. Back in 1949, the cryptographer Claude Shannon launched the idea of a one-player chess game on a computer.

Realistic Image Normalization for Multi-Domain Segmentation Machine Learning

Image normalization is a building block in medical image analysis. Conventional approaches are customarily utilized on a per-dataset basis. This strategy, however, prevents the current normalization algorithms from fully exploiting the complex joint information available across multiple datasets. Consequently, ignoring such joint information has a direct impact on the performance of segmentation algorithms. This paper proposes to revisit the conventional image normalization approach by instead learning a common normalizing function across multiple datasets. Jointly normalizing multiple datasets is shown to yield consistent normalized images as well as an improved image segmentation. To do so, a fully automated adversarial and task-driven normalization approach is employed as it facilitates the training of realistic and interpretable images while keeping performance on-par with the state-of-the-art. The adversarial training of our network aims at finding the optimal transfer function to improve both the segmentation accuracy and the generation of realistic images. We evaluated the performance of our normalizer on both infant and adult brains images from the iSEG, MRBrainS and ABIDE datasets. Results reveal the potential of our normalization approach for segmentation, with Dice improvements of up to 57.5% over our baseline. Our method can also enhance data availability by increasing the number of samples available when learning from multiple imaging domains.

How 'Microsoft Flight Simulator' became a 'living game' with Azure AI


Microsoft Flight Simulator is a triumph, one that fully captures the meditative experience of soaring through the clouds. But to bring the game to life, Microsoft and developer Asobo Studio needed more than an upgraded graphics engine to make its planes look more realistic. They needed a way to let you believably fly anywhere on the planet, with true-to-life topography and 3D models for almost everything you see, something that's especially difficult in dense cities. A task like that would be practically impossible to accomplish by hand. But it's the sort of large-scale data processing that Microsoft's Azure AI was built for.

Can We Really Cheat Death by Downloading Our Brains?


Last October, Jay Richards, author of The Human Advantage, caught up with Bradley Center director Robert J. Marks, a Baylor University computer engineering prof, at COSM 2019 to ask, what are our cheat-death chances? They were responding to futurist Ray Kurzweil's heady claims made at the conference that we will merge with computers by 2045 and live on as AI. Richards and Marks reflected on Kurzweil's claims and the thoughts of the panel responding to them. Jay Richards: He's (Kurzweil, below right) very much a sort of, I'd say, a techno-optimist. And in fact, he sort of thinks we're going to get brain scans and upload ourselves, whereas the panel… Though I know there was a diversity of opinion among the panelists, nevertheless, there was, I thought, a strong dose of realism.