This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word reinforcement.''

If you would like to add your publication to the site under one of these categories, please send e-mail with the relevant information to Bruno Castro da Silva.

Reinforcement learning is an area of machine learning which addresses how an autonomous agent can learn long-term successful behavior through interaction with its environment. The term reinforcement learning has its roots in behavioral psychology, in particular to Pavlovian models of reward learning in animals. The modern theory of reinforcement learning, however, is much more influenced by mathematical theories of optimal control in operations research, such as dynamic programming.

The key distinguishing feature of RL methods is that they learn policies indirectly, by instead learning value functions. RL methods can be constrasted with direct optimization methods, such as genetic algorithms (GA), which attempt to search the policy space directly.

Reinforcement learning is a technique, common in computer science, in which a computer system learns how best to solve some problem through trial-and-error. Classic applications of reinforcement learning involve problems as diverse as robot navigation, network administration and automated surveillance. At the Association for Uncertainty in Artificial Intelligence's annual conference this summer, researchers from MIT's Laboratory for Information and Decision Systems (LIDS) and Computer Science and Artificial Intelligence Laboratory will present a new reinforcement-learning algorithm that, for a wide range of problems, allows computer systems to find solutions much more efficiently than previous algorithms did. The paper also represents the first application of a new programming framework that the researchers developed, which makes it much easier to set up and run reinforcement-learning experiments. Alborz Geramifard, a LIDS postdoc and first author of the new paper, hopes that the software, dubbed RLPy (for reinforcement learning and Python, the programming language it uses), will allow researchers to more efficiently test new algorithms and compare algorithms' performance on different tasks.

Every year there is a brand new reinforcement learning competition. This usually consists of new organizers, and a new website! Instead of replacing the old website every year and breaking hundreds of links, we use a different subdomain each year. So, this page will always exist at: http://rl-competition.org And the specific websites for different years are: NIPS Reinforcement Learning Workshop: Benchmarks and Bakeoffs NIPS Reinforcement Learning Workshop: Benchmarks and Bakeoffs II ICML Reinforcement Learning and Benchmarking Event NIPS Workshop: The First Annual Reinforcement Learning Competition The 2008 Reinforcement Learning Competition:: http://2008.rl-competition.org

RLAI research is research directed toward the long-standing goals of AI (understanding the mind, reproducing human abilities) and is based on reinforcement learning ideas (learning from and while interacting with the world). RLAI research does not include all that is currently thought of as AI research, only that which addresses problems or issues that people regularly encounter in their everyday lives. Similarly, RLAI research does not include research that uses RL methods to solve problems that people do not face and excel at. There is a delimited and fruitful area of research at the confluence of the most ambitious goals of AI and the solution ideas that are arising from RL research.

This course gives a mathematical introduction to neural coding and dynamics. Topics include convolution, correlation, linear systems, game theory, signal detection theory, probability theory, information theory, and reinforcement learning. Applications to neural coding, focusing on the visual system are covered, as well as Hodgkin-Huxley and other related models of neural excitability, stochastic models of ion channels, cable theory, and models of synaptic transmission.

Both generative adversarial networks (GAN) in unsupervised learning and actor-critic methods in reinforcement learning (RL) have gained a reputation for being difficult to optimize. Practitioners in both fields have amassed a large number of strategies to mitigate these instabilities and improve training. Here we show that GANs can be viewed as actor-critic methods in an environment where the actor cannot affect the reward. We review the strategies for stabilizing training for each class of models, both those that generalize between the two and those that are particular to that model. We also review a number of extensions to GANs and RL algorithms with even more complicated information flow. We hope that by highlighting this formal connection we will encourage both GAN and RL communities to develop general, scalable, and stable algorithms for multilevel optimization with deep networks, and to draw inspiration across communities.

An important problem for HCI researchers is to estimate the parameter values of a cognitive model from behavioral data. This is a difficult problem, because of the substantial complexity and variety in human behavioral strategies. We report an investigation into a new approach using approximate Bayesian computation (ABC) to condition model parameters to data and prior knowledge. As the case study we examine menu interaction, where we have click time data only to infer a cognitive model that implements a search behaviour with parameters such as fixation duration and recall probability. Our results demonstrate that ABC (i) improves estimates of model parameter values, (ii) enables meaningful comparisons between model variants, and (iii) supports fitting models to individual users. ABC provides ample opportunities for theoretical HCI research by allowing principled inference of model parameter values and their uncertainty.