Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) are one of the most prominent tools for learning complicated distributions. However, problems such as mode collapse and catastrophic forgetting, prevent GAN from learning the target distribution. These problems are usually studied independently from each other. In this paper, we show that both problems are present in GAN and their combined effect makes the training of GAN unstable. We also show that methods such as gradient penalties and momentum based optimizers can improve the stability of GAN by effectively preventing these problems from happening. Finally, we study a mechanism for mode collapse to occur and propagate in feedforward neural networks.

We introduce a deep generative model for functions. Our model provides a joint distribution p(f, z) over functions f and latent variables z which lets us efficiently sample from the marginal p(f) and maximize a variational lower bound on the entropy H(f). We can thus maximize objectives of the form E_{f~p(f)}[R(f)] + c*H(f), where R(f) denotes, e.g., a data log-likelihood term or an expected reward. Such objectives encompass Bayesian deep learning in function space, rather than parameter space, and Bayesian deep RL with representations of uncertainty that offer benefits over bootstrapping and parameter noise. In this short paper we describe our model, situate it in the context of prior work, and present proof-of-concept experiments for regression and RL.

Noncritical soft-faults and model deviations are a challenge for Fault Detection and Diagnosis (FDD) of resident Autonomous Underwater Vehicles (AUVs). Such systems may have a faster performance degradation due to the permanent exposure to the marine environment, and constant monitoring of component conditions is required to ensure their reliability. This works presents an evaluation of Recurrent Neural Networks (RNNs) for a data-driven fault detection and diagnosis scheme for underwater thrusters with empirical data. The nominal behavior of the thruster was modeled using the measured control input, voltage, rotational speed and current signals. We evaluated the performance of fault classification using all the measured signals compared to using the computed residuals from the nominal model as features.

Gated recurrent neural networks have achieved remarkable results in the analysis of sequential data. Inside these networks, gates are used to control the flow of information, allowing to model even very long-term dependencies in the data. In this paper, we investigate whether the original gate equation (a linear projection followed by an element-wise sigmoid) can be improved. In particular, we design a more flexible architecture, with a small number of adaptable parameters, which is able to model a wider range of gating functions than the classical one. To this end, we replace the sigmoid function in the standard gate with a non-parametric formulation extending the recently proposed kernel activation function (KAF), with the addition of a residual skip-connection. A set of experiments on sequential variants of the MNIST dataset shows that the adoption of this novel gate allows to improve accuracy with a negligible cost in terms of computational power and with a large speed-up in the number of training iterations.

Editor's note: This is one of a series of posts which act as a collection of a set of fantastic notes on the fast.ai The author of all of these notes, Hiromi Suenaga -- which, in sum, are a great supplement review material for the course or a standalone resource in their own right -- wanted to ensure that sufficient credit was given to course creators Jeremy Howard and Rachel Thomas in these summaries. Below you will find links to the posts in this particular series, along with an excerpt from each post. Find more of Hiromi's notes here. These notes will continue to be updated and improved as I continue to review the course to "really" understand it.

WHEN TAY MADE HER DEBUT in March 2016, Microsoft had high hopes for the artificial intelligence–powered "social chatbot." Like the automated, text-based chat programs that many people had already encountered on e-commerce sites and in customer service conversations, Tay could answer written questions; by doing so on Twitter and other social media, she could engage with the masses. But rather than simply doling out facts, Tay was engineered to converse in a more sophisticated way--one that had an emotional dimension. She would be able to show a sense of humor, to banter with people like a friend. Her creators had even engineered her to talk like a wisecracking teenage girl. When Twitter users asked Tay who her parents were, she might respond, "Oh a team of scientists in a Microsoft lab. They're what u would call my parents." If someone asked her how her day had been, she could quip, "omg totes exhausted."

This meetup will be hosted jointly with Boulder Deep Learning. The format will be meet and greet at start and end, with a series of shorter lighting-talk length presentations in the middle. The goal is to give everyone a chance to find out what the local machine learning and data science communities are up to, where you can get involved, what other events are going on, etc. If you're interested in giving one of these talks, please reach out to me (Ben). I've had several people express interest.

Using more than three million photographs from the citizen science project Snapshot Serengeti, researchers trained a deep learning algorithm to automatically identify, count and describe animals in their natural habitats. Results showed the system was able to automate the process for up to 99.3 percent of images as accurately as human volunteers.

Recommender Systems are among the most common forms of Machine Learning that users will encounter, whether they're aware of it or not. It powers curated timelines on Facebook and Twitter, and "suggested videos" on YouTube. Previously formulated as a matrix factorization problem that attempts to predict a movie's ratings for a particular user, many are now approaching this problem using deep learning; the intuition is that non-linear combinations of features may yield a better prediction than a traditional matrix factorization approach can. In 2016, Covington, Adams, and Sargin demonstrated the benefits of this approach with "Deep Neural Networks for YouTube Recommendations", making Google one of the first companies to deploy production-level deep neural networks for recommender systems. Given that YouTube is the second most visited website in the United States, with over 400 hours of content uploaded per minute, recommending fresh content poses no straightforward task.

Audi plans to make five new-energy vehicle models in China by 2022, the company's China head Joachim Wedler said today. Autonomous Intelligent Driving (AID) is a wholly owned subsidiary of Audi AG with a fleet of test vehicles that are running an autonomous vehicle simulation platform from Cognata, an Israeli artificial intelligence (AI) and deep learning company. Their platform uses AI, deep learning and computer vision in a realistic and safe simulation environment to simulate and validate autonomous vehicles prior vehicles moving to test phases on real roads. "Previously autonomous vehicles were a hyped AI experiment and companies were at the stage of trying out prototypes on the road. Autonomous vehicles have now evolved into a more mature product that demands human-like performance, which means zero tolerance for safety issues and efficiency in a high traffic environment," said Danny Atsmon, CEO, Cognata.