In the first blog post of this series we introduced the topic of machine learning and discussed why there is a lot of excitement around the topic. In this blog we explore different types of machine learning. Let's start with a simple example that everyone can relate to. You want to teach a three year old some basic discipline of keeping their toys in the right place. The room is full of interlocking blocks and soft toys.

LONDON: Technological developments will dramatically change the role of agency strategists as they move from a free-associating, subjective approach to a more empirical, objective and advisory role, a leading industry figure has said. Writing in the current issue of Admap, Mark Holden, Worldwide Strategy and Planning Director at PHD, outlines the future direction of strategy that will start to emerge once attribution modelling and demand-side platforms come together. Currently, users log in to the former to pull out insights and then log in to the latter to execute their strategies. "When they are finally joined up, this will create the first closed system our industry has ever experienced," Holden says, "with this the basis into which we can drop a reinforcement learning algorithm." The point about reinforcement learning – an emerging area of artificial intelligence – is that it requires a closed system, where action and outcome are inextricably linked, in order to further develop.

Open AI, a non-profit artificial intelligence research company backed by Elon Musk, launched a toolkit for developing and comparing reinforcement learning algorithms. OpenAI Gym is a suite of environments that include simulated robotic tasks and Atari games as well as a website for people to post their results and share code. OpenAI researcher John Schulman shared some details about his organization, why reinforcement learning is important and how the OpenAI Gym will make it easier for AI researchers to design, iterate and improve their next generation applications.

Q-learning is a simple and powerful tool in solving dynamic problems where environments are unknown. It uses a balance of exploration and exploitation to find an optimal solution to the problem. In this paper, we propose using four basic emotions: joy, sadness, fear, and anger to influence a Qlearning agent. Simulations show that the proposed affective agent requires lesser number of steps to find the optimal path. We found when affective agent finds the optimal path, the ratio between exploration to exploitation gradually decreases, indicating lower total step count in the long run

If you would like to clarify the clouds over what machine learning is please check this article from Stanford first. Page 9 and 15 have good description on the artificial intelligence topic along with description of what machine learning and reinforcement learning means. If you would like to refresh your knowledge on project haystack, Berkeley study uses Haystack information circa 2012 is very helpful overview. Most updated info is also available on project-haystack.org Would like to hear opinion below to further the conversation on this complex topic.

This paper reports applications of Difference of Convex functions (DC) programming to Learning from Demonstrations (LfD) and Reinforcement Learning (RL) with expert data. This is made possible because the norm of the Optimal Bellman Residual (OBR), which is at the heart of many RL and LfD algorithms, is DC. Improvement in performance is demonstrated on two specific algorithms, namely Reward-regularized Classification for Apprenticeship Learning (RCAL) and Reinforcement Learning with Expert Demonstrations (RLED), through experiments on generic Markov Decision Processes (MDP), called Garnets.

This is a long overdue blog post on Reinforcement Learning (RL). You may have noticed that computers can now automatically learn to play ATARI games (from raw game pixels!), they are beating world champions at Go, simulated quadrupeds are learning to run and leap, and robots are learning how to perform complex manipulation tasks that defy explicit programming. It turns out that all of these advances fall under the umbrella of RL research. I also became interested in RL myself over the last year: I worked through Richard Sutton's book, read through David Silver's course, watched John Schulmann's lectures, wrote an RL library in Javascript, over the summer interned at DeepMind working in the DeepRL group, and most recently pitched in a little with the design/development of OpenAI Gym, a new RL benchmarking toolkit. So I've certainly been on this funwagon for at least a year but until now I haven't gotten around to writing up a short post on why RL is a big deal, what it's about, how it all developed and where it might be going. It's interesting to reflect on the nature of recent progress in RL. Similar to what happened in Computer Vision, the progress in RL is not driven as much as you might reasonably assume by new amazing ideas. In Computer Vision, the 2012 AlexNet was mostly a scaled up (deeper and wider) version of 1990's ConvNets. Similarly, the ATARI Deep Q Learning paper from 2013 is an implementation of a standard algorithm (Q Learning with function approximation, which you can find in the standard RL book of Sutton 1998), where the function approximator happened to be a ConvNet. AlphaGo uses policy gradients with Monte Carlo Tree Search (MCTS) - these are also standard components.

My 2 month summer internship at Skymind (the company behind the open source deeplearning library DL4J) comes to an end and this is a post to summarize what I have been working on: Building a deep reinforcement learning library for DL4J: … (drums roll) … RL4J! This post begins by an introduction to reinforcement learning and is then followed by a detailed explanation of DQN (Deep Q-Network) for pixel inputs and is concluded by an RL4J example. I will assume from the reader some familiarity with neural networks. But first, lets talk about the core concepts of reinforcement learning. A "simple aspect of science" may be defined as one which, through good fortune, I happen to understand. Reinforcement Learning is an exciting area of machine learning. It is basically the learning of an efficient strategy in a given environment. Informally, this is very similar to Pavlovian conditioning: you assign a reward for a given behavior and over time, the agents learn to reproduce that behavior in order to receive more rewards. It is an iterative trial and error process. Formally, an environment is defined as a Markov Decision Process (MDP). Note: It is usually more convenient to use the set of Action \(A_s\) which is the set of available move from a given state, than the complete set A. \(A_s\) is simply the elements \(a\) in \(A\) such that \(P(s' s, a) 0\).

Reinforcement learning framework integrated with deeplearning4j. This is a tech preview and distributed as is.

Deep Reinforcement Learning has recently gained a lot of traction in the machine learning community due to the significant amount of progress that has been made in the past few years. Traditionally, reinforcement learning algorithms were constrained to tiny, discretized grid worlds, which seriously inhibited them from gaining credibility as being viable machine learning tools. Here's a classic example from Richard Sutton's book, which I will be referencing a lot. After Deep Q-Networks [4] became a hit, people realized that deep learning methods could be used to solve high-dimensional problems. One of the subsequent challenges that the reinforcement learning community faced was figuring out how to deal with continuous action spaces.