Discriminator


Deep Learning Research Review Week 1: Generative Adversarial Nets

@machinelearnbot

This week, I'll be doing a new series called Deep Learning Research Review. The way the authors combat this is by using multiple CNN models to sequentially generate images in increasing scales. The approach the authors take is training a GAN that is conditioned on text features created by a recurrent text encoder (won't go too much into this, but here's the paper for those interested). In order to create these versatile models, the authors train with three types of data: {real image, right text}, {fake image, right text}, and {real image, wrong text}.


GANGogh: Creating Art with GANs – Towards Data Science – Medium

@machinelearnbot

StackGAN uses feature information retrieved from a Recurrent Neural Net and a two phase image generation process -- -the first phase creates a low resolution image from a noise vector and the second phase uses an encoding of the first image to create high resolution image. They also both use gated multiplicative activation functions which seem to mesh well with this global conditioning (van den Oord 2016; van den Oord 2016). In papers such as A Neural Algorithm of Artistic Style, deep learning nets learn to 1) differentiate the style of a piece of art from its content and 2) to apply that style to other content representations. We could enforce this metric by adding a penalizing term to our discriminator's cost function that tries to minimize the cross-entropy in its prediction of genre versus the real genre of a given painting, and adding a penalizing term to our generator that tries to minimize the cross-entropy of the discriminator's prediction versus the genre it was instructed to make based on the conditioning vector.


Deep Learning, Generative Adversarial Networks & Boxing – Toward a Fundamental Understanding

@machinelearnbot

Let's break down a GAN into its basic components: The overall goal of a standard GAN is to train a generator that generates diverse data samples from the true data distribution, leading to a discriminator that can only classify images as real/generated with a 50/50 guess. In the process of training this network, both the generator and the discriminator learn powerful, hierarchical representations of the underlying data that can then transfer to a variety of specific tasks like classification, segmentation, etc… and use-cases. Now that we have a fundamental understanding of GANs, let's revisit their purpose: to learn powerful representations from unlabelled data (i.e. After training a GAN, most current methods use the discriminator as a base model for transfer learning and the fine-tuning of a production model, or the generator as a source of data that is used to train a production model.


Deep adversarial learning is finally ready and will radically change the game

#artificialintelligence

Given raw data, a question to ask the network, and an objective function to evaluate the network's answer, a network learns to optimally represent (abstract) this data. As opposed to the classical deep learning approach where questions that are expected to be relevant to the task-at-hand are manually identified, and hand-crafted objective functions guide the optimization of our networks towards learning the corresponding answers. With the goal of modeling the true data distribution, the generator learns to generate realistic samples of data while the discriminator learns to determine if these samples are real or not. Improved training of Wasserstein GANs enables very stable GAN training by penalizing the norm of the gradient of the critic with respect to its input instead of clipping weights.


A Primer in Adversarial Machine Learning – The Next Advance in AI

@machinelearnbot

Even more concerning, researchers have shown that completely random nonsense images can be misclassified by CNNs with very high confidence as objects recognizable to humans, even though a human would clearly recognize that there was no image there at all (e.g. If those system observations are intentionally tainted with noise designed to defeat the CNN recognition, the system will be trained to make incorrect conclusions about whether a malevolent intrusion is occurring. Adversarial Machine Learning is an emerging area in deep neural net (DNN) research. The current state of AI has advanced to general image, text, and speech recognition, and tasks like steering the car or winning a game of chess.


The Strange Loop in Deep Learning – Intuition Machine – Medium

#artificialintelligence

My first recollection of an effective Deep Learning system that used feedback loops where in "Ladder Networks". In an architecture developed by Stanford called "Feedback Networks", the researchers explored a different kind of network that feeds back into itself and develops the internal representation incrementally: In an even more recently published research (March 2017) from UC Berkeley have created astonishingly capable image to image translations using GANs and a novel kind of regularization. The major difficulty of training Deep Learning systems has been the lack of labeled data. So the next time you see some mind boggling Deep Learning results, seek to find the strange loops that are embedded in the method.


The Strange Loop in Deep Learning – Intuition Machine – Medium

#artificialintelligence

My first recollection of an effective Deep Learning system that used feedback loops where in "Ladder Networks". The discriminator network attempts to perform a classification against data that the generative network is creating. In an architecture developed by Stanford called "Feedback Networks", the researchers explored a different kind of network that feeds back into itself and develops the internal representation incrementally: In an even more recently published research (March 2017) from UC Berkeley have created astonishingly capable image to image translations using GANs and a novel kind of regularization. So the next time you see some mind boggling Deep Learning results, seek to find the strange loops that are embedded in the method.


Deep Stubborn Networks – A Breakthrough Advance Towards Adversarial Machine Intelligence

@machinelearnbot

Generative Adversarial Networks (GANs) are one of the most exciting advancements in machine learning of the past decade. GANS were first outlined in 2014 by Ian Goodfellow et al., and have gone on to be lauded as one of the most important developments in deep learning by machine learning guru Yann LeCun. The exciting announcement yesterday of Deep Stubborn Networks (StubNets) introduces an innovative refinement to GANs, taking their development in a new direction. Geoffrey Hinton has stated that he believes Deep Stubborn Networks are a big step towards right-brain artificial general intelligence.


Deep Learning Research Review: Generative Adversarial Nets

@machinelearnbot

I briefly mentioned Ian Goodfellow's Generative Adversarial Network paper in one of my prior blog posts, 9 Deep Learning Papers You Should Know About. The way the authors combat this is by using multiple CNN models to sequentially generate images in increasing scales. The generator takes in an input noise vector from a distribution and outputs an image. The change is the traditional GAN structure is that instead of having just one generator CNN that creates the whole image, we have a series of CNNs that create the image sequentially by slowly increasing the resolution (aka going along the pyramid) and refining images in a coarse to fine fashion.


Generative Models

@machinelearnbot

However, as you might imagine, the network has millions of parameters that we can tweak, and the goal is to find a setting of these parameters that makes samples generated from random codes look like the training data. Here we introduce a second discriminator network (usually a standard convolutional neural network) that tries to classify if an input image is real or generated. Our goal then is to find parameters that produce a distribution that closely matches the true data distribution (for example, by having a small KL divergence loss). Our CIFAR-10 samples also look very sharp - Amazon Mechanical Turk workers can distinguish our samples from real data with an error rate of 21.3% (50% would be random guessing): In addition to generating pretty pictures, we introduce an approach for semi-supervised learning with GANs that involves the discriminator producing an additional output indicating the label of the input.