Every time DeepMind publishes a new paper, there is frenzied media coverage around it. Often you will read phrases that are often misleading. DeepMind Develops a Neural Network That Can Make Sense of Objects Around It. This is not only misleading, but it also makes the everyday non PhD person intimidated. In this post I will go through the paper in an attempt to explain this new architecture in simple terms.

DeepMind, a British artificial intelligence firm acquired by Google in 2014, is building an AI capable of "imagination" and understanding the consequences of previous actions. In two research papers submitted last week, DeepMind describes how the AI would be able to "construct a plan" and remember information that may be important in the future. "What differentiates these agents is that they learn a model of the world from noisy sensory data, rather than rely on privileged information such as a pre-specified, accurate simulator," said the DeepMind research team to Wired. "Imagination-based approaches are particularly helpful in situations where the agent is in a new situation and has little direct experience to rely on, or when its actions have irreversible consequences and thinking carefully is desirable over spontaneous action." Like most of DeepMind's research, it used video games to test the AI's proficiency.

How is deep learning affecting sports? I don't know about you, but I was not the most athletic kid growing up. It took me forever to make a jump shot well. When I started playing golf after college my short game was an absolute disaster. I always had a hard time visualizing what I needed to do differently. Having a coach watch and tell me what to do never seemed to do the trick.

In general there is no guarantee that, even with a lot of data, deep learning does better than other techniques, for example tree-based such as random forest or boosted trees. Do I need some Skynet to run it?

The artificial neural network is a popular framework in machine learning. To empower individual neurons, we recently suggested that the current type of neurons could be upgraded to 2nd order counterparts, in which the linear operation between inputs to a neuron and the associated weights is replaced with a nonlinear quadratic operation. A single 2nd order neurons already has a strong nonlinear modeling ability, such as implementing basic fuzzy logic operations. In this paper, we develop a general backpropagation (BP) algorithm to train the network consisting of 2nd-order neurons. The numerical studies are performed to verify of the generalized BP algorithm.

Deep networks have been successfully applied to learn transferable features for adapting models from a source domain to a different target domain. In this paper, we present joint adaptation networks (JAN), which learn a transfer network by aligning the joint distributions of multiple domain-specific layers across domains based on a joint maximum mean discrepancy (JMMD) criterion. Adversarial training strategy is adopted to maximize JMMD such that the distributions of the source and target domains are made more distinguishable. Learning can be performed by stochastic gradient descent with the gradients computed by back-propagation in linear-time. Experiments testify that our model yields state of the art results on standard datasets.

Since computers can automate such processes, automatic music generation has become a small, but steadily emerging field in Artificial Intelligence and Machine Learning. Nevertheless, automatic music generation as a problem is far from solved: musical outputs created by artificial systems are regarded as a curiosity by human listeners at best, but all too often they are taken as a direct offense to our sense of musical aesthetics. This sensitivity to violations of even the most subtle musical norms illustrates how complex the problem of (especially polyphonic) music generation is. In addition, there are hardly any objective evaluation criteria to rigorously test and compare music generation systems. This is lamentable, not least since successful methods for automatic music generation would be of considerable commercial interest to the music, gaming, and film industries.

Reinforcement learning is concerned with identifying reward-maximizing behaviour policies in environments that are initially unknown. State-of-the-art reinforcement learning approaches, such as deep Q-networks, are model-free and learn to act effectively across a wide range of environments such as Atari games, but require huge amounts of data. Model-based techniques are more data-efficient, but need to acquire explicit knowledge about the environment. In this paper, we take a step towards using model-based techniques in environments with a high-dimensional visual state space by demonstrating that it is possible to learn system dynamics and the reward structure jointly. Our contribution is to extend a recently developed deep neural network for video frame prediction in Atari games to enable reward prediction as well. To this end, we phrase a joint optimization problem for minimizing both video frame and reward reconstruction loss, and adapt network parameters accordingly. Empirical evaluations on five Atari games demonstrate accurate cumulative reward prediction of up to 200 frames. We consider these results as opening up important directions for model-based reinforcement learning in complex, initially unknown environments.

When encountering novel objects, humans are able to infer a wide range of physical properties such as mass, friction and deformability by interacting with them in a goal driven way. This process of active interaction is in the same spirit as a scientist performing experiments to discover hidden facts. Recent advances in artificial intelligence have yielded machines that can achieve superhuman performance in Go, Atari, natural language processing, and complex control problems; however, it is not clear that these systems can rival the scientific intuition of even a young child. In this work we introduce a basic set of tasks that require agents to estimate properties such as mass and cohesion of objects in an interactive simulated environment where they can manipulate the objects and observe the consequences. We found that deep reinforcement learning methods can learn to perform the experiments necessary to discover such hidden properties. By systematically manipulating the problem difficulty and the cost incurred by the agent for performing experiments, we found that agents learn different strategies that balance the cost of gathering information against the cost of making mistakes in different situations. We also compare our learned experimentation policies to randomized baselines and show that the learned policies lead to better predictions.

For many years we've been helping clients with the application of Artificial Intelligence and Machine Learning. While it can appear complex and intimidating, it's not so bad once you understand a few key concepts. Artificial Intelligence (AI) is a sector by buzzwords and hype -- so how do you cut through the noise? Think of AI as the superset -- and everything else is a subset of it. Put another way, AI is the universe and things like Machine Learning, Neural Networks, and Deep Learning is the solar systems that it's made up of.