In this paper we study how to learn stochastic, multimodal transition dynamics in reinforcement learning (RL) tasks. We focus on evaluating transition function estimation, while we defer planning over this model to future work. Stochasticity is a fundamental property of many task environments. However, discriminative function approximators have difficulty estimating multimodal stochasticity. In contrast, deep generative models do capture complex high-dimensional outcome distributions. First we discuss why, amongst such models, conditional variational inference (VI) is theoretically most appealing for model-based RL. Subsequently, we compare different VI models on their ability to learn complex stochasticity on simulated functions, as well as on a typical RL gridworld with multimodal dynamics. Results show VI successfully predicts multimodal outcomes, but also robustly ignores these for deterministic parts of the transition dynamics. In summary, we show a robust method to learn multimodal transitions using function approximation, which is a key preliminary for model-based RL in stochastic domains.

Well, computer scientists from the University of British Columbia and National University of Singapore just did that with a bipedal computer model (read: essentially a pair of animated legs) -- only instead of a cute cartoon rabbit, the teacher is a deep reinforcement learning artificial intelligence algorithm. Google's DeepMind, for example, has used reinforcement learning to teach an AI to play classic video games by working out how to achieve high scores. It's like watching your kid grow up -- except that, you know, in this case, your kid is a pair of disembodied AI legs powered by Skynet! A paper describing the work, titled "DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning" was published in the journal Transactions on Graphics.

Well, computer scientists from the University of British Columbia and National University of Singapore just did that with a bipedal computer model (read: essentially a pair of animated legs) -- only instead of a cute cartoon rabbit, the teacher is a deep reinforcement learning artificial intelligence algorithm. Google's DeepMind, for example, has used reinforcement learning to teach an AI to play classic video games by working out how to achieve high scores. It's like watching your kid grow up -- except that, you know, in this case, your kid is a pair of disembodied AI legs powered by Skynet! A paper describing the work, titled "DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning" was published in the journal Transactions on Graphics.

A bunch of people have been asking for an implementation of Deep Reinforcement Learning from Human Preferences [Christiano et al., 2017] that came out last month. This contains a simplified system designed to be easy to read and understand, plus the webapp that we used for collecting feedback from humans. Happy to answer any questions that you have here!

Wireless systems perform rate adaptation to transmit at highest possible instantaneous rates. Rate adaptation has been increasingly granular over generations of wireless systems. The base-station uses SINR and packet decode feedback called acknowledgement/no acknowledgement (ACK/NACK) to perform rate adaptation. SINR is used for rate anchoring called inner look adaptation and ACK/NACK is used for fine offset adjustments called Outer Loop Link Adaptation (OLLA). We cast the OLLA as a reinforcement learning problem of the class of Multi-Armed Bandits (MAB) where the different offset values are the arms of the bandit. In OLLA, as the offset values increase, the probability of packet error also increase, and every user equipment (UE) has a desired Block Error Rate (BLER) to meet certain Quality of Service (QoS) requirements. For this MAB we propose a binary search based algorithm which achieves a Probably Approximately Correct (PAC) solution making use of bounds from large deviation theory and confidence bounds. In addition to this we also discuss how a Thompson sampling or UCB based method will not help us meet the target objectives. Finally, simulation results are provided on an LTE system simulator and thereby prove the efficacy of our proposed algorithm.

In this work, we design and evaluate a computational learning model that enables a human-robot team to co-develop joint strategies for performing novel tasks that require coordination. The joint strategies are learned through "perturbation training," a human team-training strategy that requires team members to practice variations of a given task to help their team generalize to new variants of that task. We formally define the problem of human-robot perturbation training and develop and evaluate the first end-to-end framework for such training, which incorporates a multi-agent transfer learning algorithm, human-robot co-learning framework and communication protocol. Our transfer learning algorithm, Adaptive Perturbation Training (AdaPT), is a hybrid of transfer and reinforcement learning techniques that learns quickly and robustly for new task variants. We empirically validate the benefits of AdaPT through comparison to other hybrid reinforcement and transfer learning techniques aimed at transferring knowledge from multiple source tasks to a single target task. We also demonstrate that AdaPT's rapid learning supports live interaction between a person and a robot, during which the human-robot team trains to achieve a high level of performance for new task variants. We augment AdaPT with a co-learning framework and a computational bi-directional communication protocol so that the robot can co-train with a person during live interaction. Results from large-scale human subject experiments (n=48) indicate that AdaPT enables an agent to learn in a manner compatible with a human's own learning process, and that a robot undergoing perturbation training with a human results in a high level of team performance. Finally, we demonstrate that human-robot training using AdaPT in a simulation environment produces effective performance for a team incorporating an embodied robot partner.

One question central to Reinforcement Learning is how to learn a feature representation that supports algorithm scaling and re-use of learned information from different tasks. Successor Features approach this problem by learning a feature representation that satisfies a temporal constraint. We present an implementation of an approach that decouples the feature representation from the reward function, making it suitable for transferring knowledge between domains. We then assess the advantages and limitations of using Successor Features for transfer.

It branches out from Artificial Intelligence and is classified as a Machine Learning type. Leveraging reinforcement learning, software agents and machines are made to ascertain the ideal behavior in a specific context with the aim of maximizing its performance. When the learning agent acts on trial and error, it is termed as exploration, and when it acts based on the knowledge gained from the environment, it is referred to as exploitation. The environment rewards the agent for correct actions, which is the reinforcement signal. Leveraging the rewards obtained, the agent improves its environment knowledge to select the next action.

The framework takes a game-like approach, accesses the system, learns about the system, and figures out how it can be attacked and how it can evade an attack. "Reinforcement learning has produced models that top human performance in a myriad of games. Using similar techniques, our PE malware evasion technique can be framed as a competitive game between our agent and the machine learning model detector. Our agent inspects a PE file and selects a sequence of functionality-preserving mutations to the PE file which best evade the malware detection model. The agent learns through the experience of thousands of "games" against the detector, which sequence of actions is most likely to result in an [invasive variant.]

The Particle Swarm Optimization Policy (PSO-P) has been recently introduced and proven to produce remarkable results on interacting with academic reinforcement learning benchmarks in an off-policy, batch-based setting. To further investigate the properties and feasibility on real-world applications, this paper investigates PSO-P on the so-called Industrial Benchmark (IB), a novel reinforcement learning (RL) benchmark that aims at being realistic by including a variety of aspects found in industrial applications, like continuous state and action spaces, a high dimensional, partially observable state space, delayed effects, and complex stochasticity. The experimental results of PSO-P on IB are compared to results of closed-form control policies derived from the model-based Recurrent Control Neural Network (RCNN) and the model-free Neural Fitted Q-Iteration (NFQ). Experiments show that PSO-P is not only of interest for academic benchmarks, but also for real-world industrial applications, since it also yielded the best performing policy in our IB setting. Compared to other well established RL techniques, PSO-P produced outstanding results in performance and robustness, requiring only a relatively low amount of effort in finding adequate parameters or making complex design decisions.