generator


Deep Learning Research Review Week 1: Generative Adversarial Nets

@machinelearnbot

This week, I'll be doing a new series called Deep Learning Research Review. The way the authors combat this is by using multiple CNN models to sequentially generate images in increasing scales. The approach the authors take is training a GAN that is conditioned on text features created by a recurrent text encoder (won't go too much into this, but here's the paper for those interested). In order to create these versatile models, the authors train with three types of data: {real image, right text}, {fake image, right text}, and {real image, wrong text}.


Taxonomy of Methods for Deep Meta Learning

#artificialintelligence

Two recent papers that were submitted to ICLR 2017 explore the use of Reinforcement learning to learn new kinds of Deep Learning architectures ("Designing Neural Network Architectures using Reinforcement Learning" and "Neural Architecture Search with Reinforcement Learning"). The second paper (Neural Architecture Search) employs uses Reinforcement Learning (RL) to train a an architecture generator LSTM to build a language that describes new DL architectures. The trained generator RNN is a two-layer LSTM, this RNN generates an architecture that is trained for 50 epochs. We have a glimpse of a DSL driven architecture in my previous post about "A Language Driven Approach to Deep Learning Training" where a prescription that is quite general is presented.


GANGogh: Creating Art with GANs – Towards Data Science – Medium

@machinelearnbot

StackGAN uses feature information retrieved from a Recurrent Neural Net and a two phase image generation process -- -the first phase creates a low resolution image from a noise vector and the second phase uses an encoding of the first image to create high resolution image. They also both use gated multiplicative activation functions which seem to mesh well with this global conditioning (van den Oord 2016; van den Oord 2016). In papers such as A Neural Algorithm of Artistic Style, deep learning nets learn to 1) differentiate the style of a piece of art from its content and 2) to apply that style to other content representations. We could enforce this metric by adding a penalizing term to our discriminator's cost function that tries to minimize the cross-entropy in its prediction of genre versus the real genre of a given painting, and adding a penalizing term to our generator that tries to minimize the cross-entropy of the discriminator's prediction versus the genre it was instructed to make based on the conditioning vector.


Deep Learning, Generative Adversarial Networks & Boxing – Toward a Fundamental Understanding

@machinelearnbot

Let's break down a GAN into its basic components: The overall goal of a standard GAN is to train a generator that generates diverse data samples from the true data distribution, leading to a discriminator that can only classify images as real/generated with a 50/50 guess. In the process of training this network, both the generator and the discriminator learn powerful, hierarchical representations of the underlying data that can then transfer to a variety of specific tasks like classification, segmentation, etc… and use-cases. Now that we have a fundamental understanding of GANs, let's revisit their purpose: to learn powerful representations from unlabelled data (i.e. After training a GAN, most current methods use the discriminator as a base model for transfer learning and the fine-tuning of a production model, or the generator as a source of data that is used to train a production model.


Machine Learning in Real Life: Tales from the Trenches to the Cloud – Part 1

#artificialintelligence

I recommend that any avid machine learning enthusiast who wants to proceed doing real life machine learning work give it a go. Also, it's important to capture user interactions: Allow your users to rate your recommendations and use other interaction data (clicks or wait times) to help improve data quality. In the next episode of Machine Learning in Real Life, I will talk about the other parts missing from my OSD: Analysis and Production. I hope that answers some of those burning questions you may have about building Machine Learning systems in real life.


This AI-Powered Web App Turns Your Doodle Into Nightmare Fuel - Geek.com

#artificialintelligence

What you see at the top there was one of my attempts, which the clearly sinister AI that powers this nightmare generator turned into some sort of donkey-earned, grimacing cyclops. To be fair, the app's real purpose is to create realistic-looking human faces from a detailed line art original. It's a machine learning tool that maps an input image to an output image based on data that's been fed to it during the learning process. Now that you've peeked into the machine learning rabbit hole, why not waste the rest of your day playing around with some other tools that can create disturbing images, like the Deep Dream Generator… or browse through MIT's Nightmare Machine creations!


6 areas of AI and Machine Learning to watch closely

@machinelearnbot

These include long-short term memory networks (a recurrent neural network variant) that are capable of processing and predicting time series, DeepMind's differentiable neural computer that combines neural networks and memory systems in order to learn from and navigate complex data structures on their own, the elastic weight consolidation algorithm that slows down learning on certain weights depending on how important they are to previously seen tasks, andprogressive neural networks that learn lateral connections between task-specific models to extract useful features from previously learned networks for a new task. Without large scale training data, deep learning models won't converge on their optimal settings and won't perform well on complex tasks such as speech recognition or machine translation. A major catalyst for progress in AI is the repurposing of graphics processing units (GPUs) for training large neural network models. This is exciting because of the clear accelerating returns AI systems deliver to their owners and users: Faster and more efficient model training better user experience user engages with the product more creates larger data set improves model performance through optimisation.


Deep adversarial learning is finally ready and will radically change the game

#artificialintelligence

Given raw data, a question to ask the network, and an objective function to evaluate the network's answer, a network learns to optimally represent (abstract) this data. As opposed to the classical deep learning approach where questions that are expected to be relevant to the task-at-hand are manually identified, and hand-crafted objective functions guide the optimization of our networks towards learning the corresponding answers. With the goal of modeling the true data distribution, the generator learns to generate realistic samples of data while the discriminator learns to determine if these samples are real or not. Improved training of Wasserstein GANs enables very stable GAN training by penalizing the norm of the gradient of the critic with respect to its input instead of clipping weights.


A Primer in Adversarial Machine Learning – The Next Advance in AI

@machinelearnbot

Even more concerning, researchers have shown that completely random nonsense images can be misclassified by CNNs with very high confidence as objects recognizable to humans, even though a human would clearly recognize that there was no image there at all (e.g. If those system observations are intentionally tainted with noise designed to defeat the CNN recognition, the system will be trained to make incorrect conclusions about whether a malevolent intrusion is occurring. Adversarial Machine Learning is an emerging area in deep neural net (DNN) research. The current state of AI has advanced to general image, text, and speech recognition, and tasks like steering the car or winning a game of chess.


The Strange Loop in Deep Learning – Intuition Machine – Medium

#artificialintelligence

My first recollection of an effective Deep Learning system that used feedback loops where in "Ladder Networks". In an architecture developed by Stanford called "Feedback Networks", the researchers explored a different kind of network that feeds back into itself and develops the internal representation incrementally: In an even more recently published research (March 2017) from UC Berkeley have created astonishingly capable image to image translations using GANs and a novel kind of regularization. The major difficulty of training Deep Learning systems has been the lack of labeled data. So the next time you see some mind boggling Deep Learning results, seek to find the strange loops that are embedded in the method.