This is a long overdue blog post on Reinforcement Learning (RL). You may have noticed that computers can now automatically learn to play ATARI games (from raw game pixels!), they are beating world champions at Go, simulated quadrupeds are learning to run and leap, and robots are learning how to perform complex manipulation tasks that defy explicit programming. It turns out that all of these advances fall under the umbrella of RL research. I also became interested in RL myself over the last year: I worked through Richard Sutton's book, read through David Silver's course, watched John Schulmann's lectures, wrote an RL library in Javascript, over the summer interned at DeepMind working in the DeepRL group, and most recently pitched in a little with the design/development of OpenAI Gym, a new RL benchmarking toolkit. So I've certainly been on this funwagon for at least a year but until now I haven't gotten around to writing up a short post on why RL is a big deal, what it's about, how it all developed and where it might be going. It's interesting to reflect on the nature of recent progress in RL. Similar to what happened in Computer Vision, the progress in RL is not driven as much as you might reasonably assume by new amazing ideas. In Computer Vision, the 2012 AlexNet was mostly a scaled up (deeper and wider) version of 1990's ConvNets. Similarly, the ATARI Deep Q Learning paper from 2013 is an implementation of a standard algorithm (Q Learning with function approximation, which you can find in the standard RL book of Sutton 1998), where the function approximator happened to be a ConvNet. AlphaGo uses policy gradients with Monte Carlo Tree Search (MCTS) - these are also standard components.

Due to the recent achievements of artificial neural networks across many different tasks (such as face recognition, object detection and Go), deep learning has become extremely popular. This post aims to be a starting point for those interested in learning more about it. If you already have a basic understanding of linear algebra, calculus, probability and programming: I recommend starting with Stanford's CS231n. The course notes are comprehensive and well-written. The slides for each lesson are also available, and even though the accompanying videos were removed from the official site, re-uploads are quite easy to find online.

Manulife's Lab of Forward Thinking is partnering with indico data solutions, a Boston-based company specializing in deep Learning, to better analyze unstructured financial data, the insurer announced today. Indico's platform will enable the Canadian insurer to evaluate data from news articles and analyst reports and recommend investment decisions to portfolio managers. Deciphering natural language and extracting insights is one of indico's platform's core strengths, according to the companies. "Indico will help us accelerate our use of deep learning to improve the decision-making capabilities of our analysts, portfolio managers and researchers," said Greg Framke, executive vice president and chief information officer, Manulife, in a statement. "By introducing new capabilities that we know will add to our user experience and overall impact, we will improve the customer experience."

My 2 month summer internship at Skymind (the company behind the open source deeplearning library DL4J) comes to an end and this is a post to summarize what I have been working on: Building a deep reinforcement learning library for DL4J: … (drums roll) … RL4J! This post begins by an introduction to reinforcement learning and is then followed by a detailed explanation of DQN (Deep Q-Network) for pixel inputs and is concluded by an RL4J example. I will assume from the reader some familiarity with neural networks. But first, lets talk about the core concepts of reinforcement learning. A "simple aspect of science" may be defined as one which, through good fortune, I happen to understand. Reinforcement Learning is an exciting area of machine learning. It is basically the learning of an efficient strategy in a given environment. Informally, this is very similar to Pavlovian conditioning: you assign a reward for a given behavior and over time, the agents learn to reproduce that behavior in order to receive more rewards. It is an iterative trial and error process. Formally, an environment is defined as a Markov Decision Process (MDP). Note: It is usually more convenient to use the set of Action \(A_s\) which is the set of available move from a given state, than the complete set A. \(A_s\) is simply the elements \(a\) in \(A\) such that \(P(s' s, a) 0\).

China's biggest search engine, Baidu, announced it will use Intel's Xeon Phi processor when the processor's release plan was disclosed on August 17 at Intel's annual developer forum in San Francisco. "When it comes to AI (artificial intelligence), Intel's Xeon Phi is a great fit," said Jing Wang, a senior vice president of Baidu, who joined Diane Bryant, executive vice president in charge of Intel's data center group, at the forum. Intel said Xeon Phi will help accelerate deep learning, a computerized technique increasingly used for tasks such as interpreting speech, identifying objects in photos and piloting autonomous vehicles. Baidu, having researched the application of artificial intelligence for years, is considering using the new chip to support its voice recognition system, called Deep Speech. Deep Speech is based on the collection of 7,000 hours of voice clips created by 9,600 people.

Deep-learning artificial intelligence has mostly relied upon the general-purpose GPU hardware used in many other computing tasks. But Intel's recent acquisition of the startup Nervana Systems will give the tech giant ownership of a specialized chip designed specifically for deep learning AI applications. That could give Intel a huge lead in the race to develop next-generation artificial intelligence capable of swiftly finding patterns in huge data sets and learning through imitation. Nervana has leaned heavily on GPU hardware to build its own portfolio of deep-learning AI services for both companies and independent developers. But the startup has also been developing its own specialized deep-learning hardware, called Nervana Engine, that includes only the components necessary for running deep-learning algorithms and eliminates the extra components used for general-purpose GPU tasks.

Financial services group Manulife is collaborating with Indico Data Solutions, a Boston-based company that specializes in «Deep Learning». Manulife's Lab of Forward Thinking (LOFT) is collaborating with Indico Data Solutions, a Boston-based company that specializes in «Deep Learning». This collaboration is part of a strategic effort to leverage best-in-class products and accelerate business adoption of innovative technologies such as Artificial Intelligence (AI), blockchain and virtual reality. Manulife's LOFT will use indico's platform to develop an AI and Deep Learning tool to analyze unstructured financial data. Using Deep Learning, Manulife will be able to analyze data from news articles, analyst reports and other similar sources and present recommendations that could help investment researchers and portfolio managers make more informed decisions faster than ever before.

I have multivariate time series data from sensors. I want to feed the raw, unlabeled data into a deep learning model (Thinking of deep belief nets right now). I hope that in the output layer of the rbm, there will be features of the time series. With every additional rbm, i will learn features of higher level . Those features will be used to represent the time series.

This blog series is based on my upcoming talk on re-usability of Deep Learning Models at the Hadoop Strata World Conference in Singapore. This blog series will be in several parts – where I describe my experiences and go deep into the reasons behind my choices. Deep learning is an emerging field of research, which has its application across multiple domains. I try to show how transfer learning and fine tuning strategy leads to re-usability of the same Convolution Neural Network model in different disjoint domains. Application of this model across various different domains brings value to using this fine-tuned model.

This is a blog series in several parts -- where I describe my experiences and go deep into the reasons behind my choices. In Part 1, I discussed the pros and cons of different symbolic frameworks, and my reasons for choosing Theano (with Lasagne) as my platform of choice. Part 2 of this blog series is based on my upcoming talk at The Data Science Conference, 2016. Here in Part 2, I describe Deep Convolutional Neural Networks (DCNNs) and how Transfer learning and Fine-tuning helps better the training process for domain specific images. Please feel free to email me at trivedianusua23@gmail.com if you have questions.