Abnormal states in deep reinforcement learning~(RL) are states that are beyond the scope of an RL policy. Such states may make the RL system unsafe and impede its deployment in real scenarios. In this paper, we propose a simple yet effective anomaly detection framework for deep RL algorithms that simultaneously considers random, adversarial and out-of-distribution~(OOD) state outliers. In particular, we attain the class-conditional distributions for each action class under the Gaussian assumption, and rely on these distributions to discriminate between inliers and outliers based on Mahalanobis Distance~(MD) and Robust Mahalanobis Distance. We conduct extensive experiments on Atari games that verify the effectiveness of our detection strategies. To the best of our knowledge, we present the first in-detail study of statistical and adversarial anomaly detection in deep RL algorithms. This simple unified anomaly detection paves the way towards deploying safe RL systems in real-world applications.

Task-oriented dialog systems are often trained on human/human dialogs, such as collected from Wizard-of-Oz interfaces. However, human/human corpora are frequently too small for supervised training to be effective. This paper investigates two approaches to training agent-bots and user-bots through self-play, in which they autonomously explore an API environment, discovering communication strategies that enable them to solve the task. We give empirical results for both reinforcement learning and game-theoretic equilibrium finding.

Reinforcement Learning (RL) has shown human-level performance in solving complex, single setting virtual environments Mnih et al. [2013] & Silver et al. [2016]. However, applications and theory in RL problems have been far less developed and it has been posed that this is due to a wide divide between the empirical methodology associated with virtual environments in RL research and the challenges associated with reality Dulac-Arnold et al. [2019]. Simply put, Text Games provide a safe and data efficient way to learn from environments that mimic language found in real-world scenarios Shridhar et al. [2020]. Natural language (NL) has been introduced as a solution to many of the challenges in RL Luketina et al. [2019], as NL can facilitate the transfer of abstract knowledge to downstream tasks. However, RL approaches on these language driven environments are still limited in their development and therefore a call has been made for an improvement on the evaluation settings where language is a first-class component. Text Games gained wider acceptance as a testbed for NL research following work Figure 1: Sample gameplay from Narasimhan et al. [2015] who leveraged the Deep Q Network (DQN) framework from a fantasy Text Game as for policy learning on a set of synthetic textual games. Text Games are both partially given by Narasimhan et al. observable (as shown in Figure 1) and include outcomes that make reward signals [2015] where the player takes simple to define, making them a suitable problem for Reinforcement Learning to the action'Go East' to cross solve. However, research so far has been performed independently, with many authors the bridge.

Volt-Var control refers to the control of voltage (Volt) and reactive power (Var) in power distribution systems to achieve healthy operation of the systems. By optimally dispatching voltage regulators, switchable capacitors, and controllable batteries, Volt-Var control helps to flatten voltage profiles and reduce power losses across the power distribution systems. It is hence rated as the most desired function for power distribution systems [Borozan et al., 2001]. The center of the Volt-Var control is an optimization for voltage profiles and power losses governed by networked constraints. Represent a power distribution system as a tree graph (N, ξ), where N is the set of nodes or buses and ξ is the set of edges or lines and transformers.

Since the development of generic language models trained on massive unlabelled text corpora (Radford et al., 2019; Brown et al., 2020), state-of-the art language processing systems rely on sequential transfer learning (Ruder, 2019). The pretrained Language Model (LM) is fine-tuned on the downstream task using a standard supervised learning (SL) objective (Wu et al., 2019; Peters et al., 2019). Yet, such an approach suffers from several issues (Chen et al., 2020): (i) catastrophic forgetting when a model forgets previously learned knowledge and overfits to target domains, (ii) computational inefficiency from fine-tuning billionparameters networks, and (iii) the need of supervised datasets. Moreover, task-specific language models learned with SL suffer from well-studied text degeneration issues (Holtzman et al., 2019), such as the exposure bias (Bengio et al., 2015), language biases (Saleh et al., 2020; Jaques et al., 2020), or a lack of diversity (Li et al., 2015). On the other hand, text generation can be naturally framed as a sequential decision making problem, with the sequence of words seen as successive actions over a vocabulary. Thus, some researchers have recently focused on learning language models using instead Reinforcement Learning (RL) (Strub et al., 2017; Das et al., 2017; Narasimhan et al., 2015).

Awesome Reinforcement Learning - A handpicked collection of Lectures, Books, Surveys, Papers / Thesis, Codes, Tutorials / Websites, Online Demos and Open Source Reinforcement Learning Platforms related to Reinforcement Learning. Machine Learning Cheat Sheets - This repository aims at summing up in the same place all the important notions that are covered in Stanford's CS 229 Machine Learning course, and include: Refreshers in related topics that highlight the key points of the prerequisites of the course and Cheatsheets for each machine learning field, as well as another dedicated to tips and tricks to have in mind when training a model. Deep Learning Papers - This repo covers the most cited papers on various topics like Image Segmentation / Object Detection, Natural Language Processing / RNNs, Reinforcement Learning / Robotics, Convolutional Network Models, Unsupervised / Generative Models, and more. Machine learning Basics - This repository contains implementations of basic machine learning algorithms in plain Python (Python Version 3.6). All algorithms are implemented from scratch without using additional machine learning libraries.

Multi-task learning ideally allows robots to acquire a diverse repertoire of useful skills. However, many multi-task reinforcement learning efforts assume the robot can collect data from all tasks at all times. In reality, the tasks that the robot learns arrive sequentially, depending on the user and the robot's current environment. In this work, we study a practical sequential multi-task RL problem that is motivated by the practical constraints of physical robotic systems, and derive an approach that effectively leverages the data and policies learned for previous tasks to cumulatively grow the robot's skill-set. In a series of simulated robotic manipulation experiments, our approach requires less than half the samples than learning each task from scratch, while avoiding impractical round-robin data collection. On a Franka Emika Panda robot arm, our approach incrementally learns ten challenging tasks, including bottle capping and block insertion.

An unimaginable amount of data is continually being generated by scientific experiments, longitudinal studies, clinical trials, and hospital records--but what can be done with all this information? Barbara Engelhardt (she/her), PhD, is building machine-learning models and statistical tools to make use of that data and find ways to better understand, and even prevent, disease. She is now joining Gladstone Institutes as a senior investigator. "Barbara is an innovator in computational biology," says Katie Pollard, PhD, director of the Gladstone Institute of Data Science and Biotechnology. "She brings vast expertise in statistical models and will help expand our machine-learning program. We're thrilled she's joining our team."

Meta-reinforcement learning (meta-RL) algorithms allow for agents to learn new behaviors from small amounts of experience, mitigating the sample inefficiency problem in RL. However, while meta-RL agents can adapt quickly to new tasks at test time after experiencing only a few trajectories, the meta-training process is still sample-inefficient. Prior works have found that in the multi-task RL setting, relabeling past transitions and thus sharing experience among tasks can improve sample efficiency and asymptotic performance. We apply this idea to the meta-RL setting and devise a new relabeling method called Hindsight Foresight Relabeling (HFR). We construct a relabeling distribution using the combination of hindsight, which is used to relabel trajectories using reward functions from the training task distribution, and foresight, which takes the relabeled trajectories and computes the utility of each trajectory for each task. HFR is easy to implement and readily compatible with existing meta-RL algorithms. We find that HFR improves performance when compared to other relabeling methods on a variety of meta-RL tasks. Deep Reinforcement Learning (RL) has achieved success on a wide variety of tasks, ranging from computer games to robotics. However, RL agents are typically trained on a single task and are extremely sample-inefficient, often requiring millions of samples to learn a good policy for just that one task. Ideally, RL agents should be able to utilize their prior knowledge and adapt to tasks quickly, just as humans do.

Over the past decades, researchers have been pushing the limits of Deep Reinforcement Learning (DRL). Although DRL has attracted substantial interest from practitioners, many are blocked by having to search through a plethora of available methodologies that are seemingly alike, while others are still building RL agents from scratch based on classical theories. To address the aforementioned gaps in adopting the latest DRL methods, I am particularly interested in testing out if any of the recent technology developed by the leads in the field can be readily applied to a class of optimal trading problems. Unsurprisingly, many prominent breakthroughs in DRL are investigated and tested on strategic games: from AlphaGo to AlphaStar and at about the same time, OpenAI Five. Thus, in this writing, I want to show precisely how to use a DRL library that is initially built for games in a fundamental trading problem; mean reversion. And by introducing a framework that incorporates economically-motivated function properties, I also demonstrate, through the library, a highly-performant and convergent DRL solution to decision-making financial problems in general.