Most learning algorithms are not invariant to the scale of the function that is being approximated. We propose to adaptively normalize the targets used in learning. This is useful in value-based reinforcement learning, where the magnitude of appropriate value approximations can change over time when we update the policy of behavior. Our main motivation is prior work on learning to play Atari games, where the rewards were all clipped to a predetermined range. This clipping facilitates learning across many different games with a single learning algorithm, but a clipped reward function can result in qualitatively different behavior. Using the adaptive normalization we can remove this domain-specific heuristic without diminishing overall performance.

Machine Learning is one of the most exciting fields in the world. Every week we discover something new, something amazing, something revolutionary. It's incredible, but it can also be overwhelming. That's why we created This Week in Machine Learning! Each week we publish a curated list of Machine Learning stories as a resource to help you keep pace with all these exciting developments.

With Google DeepMind's recent success in mastering the game of Go, Tesla's advances in autonomous driving capabilities, and voice recognition systems like Amazon's Alexa taking off, interest in AI and machine learning have reached an all-time high. Those living in the AI world in the 1980s remember what has been referred to as an "AI winter"--a time when the inflated expectations resulted in a "crash," and funding began to dry up. While it's unlikely that the current enthusiasm in AI will wane, some worry that huge attention, and expectations, about AI could have negative side effects. Some also worry about how AI is equated with machine learning--or even, more specifically, deep learning, which is a narrow subset of AI. So, what happened in the '80s?

This thesis uses deep learning algorithms to forecast financial data. The deep learning framework is used to train a neural network. The deep neural network is a Deep Belief Network (DBN) coupled to a Multilayer Perceptron (MLP). It is used to choose stocks to form portfolios. The portfolios have better returns than the median of the stocks forming the list. The stocks forming the S&P 500 are included in the study. The results obtained from the deep neural network are compared to benchmarks from a logistic regression network, a multilayer perceptron and a naive benchmark. The results obtained from the deep neural network are better and more stable than the benchmarks. The findings support that deep learning methods will find their way in finance due to their reliability and good performance.

Summary: The most important developments in Deep Learning and AI in the last year may not be technical at all, but rather a major change in business model. In the space of about six months all the majors have made their Deep Learning IP open source, hoping to gain on the competition from the power of the broader developer base and wide adoption. To say that the last year has been big for Deep Learning is an understatement. There have been some spectacular technical innovations like Microsoft winning the ImageNet competition with a neural net comprised of 152 layers (where 6 or 7 layers is more the norm). But the big action especially in the last six months has been in the business model for Deep Learning.

Machine learning companies are being snapped up in droves by tech giants cognizant that these startups represent a new wave of technology innovation. This month alone, Intel announced plans to acquire deep learning startup Nervana Systems. Apple confirmed it would acquire Turi. Earlier this year, Twitter acquired Magic Pony Technology, Salesforce acquired PredictionIO, ESI Group acquired Mineset, and Apple acquired Emotient, among other deals. PricewaterhouseCoopers said 29 machine learning companies have been acquired so far this year by companies large and small, and total deals in 2016 will likely exceed the 37 such buyouts made last year.

The world's leading non-profit artificial intelligence research team needs the world's fastest AI system. "I thought it was incredibly appropriate that the world's first supercomputer dedicated to artificial intelligence would go to the laboratory that was dedicated to open artificial intelligence," Huang said. OpenAI's researchers will put the first production DGX-1 -- packing 170 teraflops of computing power, equal to 250 conventional servers -- to work on artificial intelligence's toughest problems. OpenAI's team is working at the cutting-edge of a field that promises incredible advances. Imagine artificial personal assistants that can coordinate our digital lives and autonomous cars and robots that are accessible to everyone.

The new machine, called a DGX-1, is optimized for the form of machine learning known as deep learning, which involves feeding data to a large network of crudely simulated neurons and has resulted in great strides in artificial intelligence in recent years. Language remains a very tricky problem for artificial intelligence, but in recent years researchers have made progress in applying deep learning to the problem (see "AI's Language Problem"). "This will allow us to train models on larger data sets, which we have found leads to progress in AI." OpenAI hopes to use reinforcement learning to build robots capable of performing useful chores around the home, although this may prove a time-consuming challenge (see "This Is the Robot Maid Elon Musk Is Funding" and "The Robot You Want Most Is Far from Reality").

Is it possible that the secret to building machine intelligence lies in spending endless hours reading Reddit? That's one question a team of researchers at OpenAI, a nonprofit backed by several Silicon Valley luminaries, hopes to answer with a new kind of supercomputer developed by chipmaker Nvidia. The researchers are also training robots do the dishes through experimentation, and they are building algorithms capable of learning to play a wide variety of different computer games. The new machine, called a DGX-1, is optimized for the form of machine learning known as deep learning, which involves feeding data to a large network of crudely simulated neurons and has resulted in great strides in artificial intelligence in recent years. The DGX-1 will let AI researchers train deep-learning systems more quickly using more data.

We are now free to share some of our perspectives on the company and its mission to accelerate the future with custom chips for deep learning. I'll share a recap of the Nervana story, from an investor's perspective, and try to explain why machine learning is of fundamental importance to every business over time. In short, I think the application of iterative algorithms (e.g., machine learning, directed evolution, generative design) to build complex systems is the most powerful advance in engineering since the Scientific Method. Machine learning allows us to build software solutions that exceed human understanding, and shows us how AI can innervate every industry. By crude analogy, Nervana is recapitulating the evolutionary history of the human brain within computing -- moving from the logical constructs of the reptilian brain to the cortical constructs of the human brain, with massive arrays of distributed memory and iterative learning algorithms. Not surprisingly, the founders integrated experiences in neuroscience, distributed computing, and networking -- a delightful mélange for tackling cognitive computing.