Deep reinforcement learning provides a promising approach for text-based games in studying natural language communication between humans and artificial agents. However, the generalization still remains a big challenge as the agents depend critically on the complexity and variety of training tasks. In this paper, we address this problem by introducing a hierarchical framework built upon the knowledge graph-based RL agent. In the high level, a meta-policy is executed to decompose the whole game into a set of subtasks specified by textual goals, and select one of them based on the KG. Then a sub-policy in the low level is executed to conduct goal-conditioned reinforcement learning. We carry out experiments on games with various difficulty levels and show that the proposed method enjoys favorable generalizability.

Network economics is the study of a rich class of equilibrium problems that occur in the real world, from traffic management to supply chains and two-sided online marketplaces. In this paper we explore causal inference in network economics, building on the mathematical framework of variational inequalities, which is a generalization of classical optimization. Our framework can be viewed as a synthesis of the well-known variational inequality formalism with the broad principles of causal inference.

Abnormal states in deep reinforcement learning~(RL) are states that are beyond the scope of an RL policy. Such states may make the RL system unsafe and impede its deployment in real scenarios. In this paper, we propose a simple yet effective anomaly detection framework for deep RL algorithms that simultaneously considers random, adversarial and out-of-distribution~(OOD) state outliers. In particular, we attain the class-conditional distributions for each action class under the Gaussian assumption, and rely on these distributions to discriminate between inliers and outliers based on Mahalanobis Distance~(MD) and Robust Mahalanobis Distance. We conduct extensive experiments on Atari games that verify the effectiveness of our detection strategies. To the best of our knowledge, we present the first in-detail study of statistical and adversarial anomaly detection in deep RL algorithms. This simple unified anomaly detection paves the way towards deploying safe RL systems in real-world applications.

Task-oriented dialog systems are often trained on human/human dialogs, such as collected from Wizard-of-Oz interfaces. However, human/human corpora are frequently too small for supervised training to be effective. This paper investigates two approaches to training agent-bots and user-bots through self-play, in which they autonomously explore an API environment, discovering communication strategies that enable them to solve the task. We give empirical results for both reinforcement learning and game-theoretic equilibrium finding.

Reinforcement Learning (RL) has shown human-level performance in solving complex, single setting virtual environments Mnih et al. [2013] & Silver et al. [2016]. However, applications and theory in RL problems have been far less developed and it has been posed that this is due to a wide divide between the empirical methodology associated with virtual environments in RL research and the challenges associated with reality Dulac-Arnold et al. [2019]. Simply put, Text Games provide a safe and data efficient way to learn from environments that mimic language found in real-world scenarios Shridhar et al. [2020]. Natural language (NL) has been introduced as a solution to many of the challenges in RL Luketina et al. [2019], as NL can facilitate the transfer of abstract knowledge to downstream tasks. However, RL approaches on these language driven environments are still limited in their development and therefore a call has been made for an improvement on the evaluation settings where language is a first-class component. Text Games gained wider acceptance as a testbed for NL research following work Figure 1: Sample gameplay from Narasimhan et al. [2015] who leveraged the Deep Q Network (DQN) framework from a fantasy Text Game as for policy learning on a set of synthetic textual games. Text Games are both partially given by Narasimhan et al. observable (as shown in Figure 1) and include outcomes that make reward signals [2015] where the player takes simple to define, making them a suitable problem for Reinforcement Learning to the action'Go East' to cross solve. However, research so far has been performed independently, with many authors the bridge.

Volt-Var control refers to the control of voltage (Volt) and reactive power (Var) in power distribution systems to achieve healthy operation of the systems. By optimally dispatching voltage regulators, switchable capacitors, and controllable batteries, Volt-Var control helps to flatten voltage profiles and reduce power losses across the power distribution systems. It is hence rated as the most desired function for power distribution systems [Borozan et al., 2001]. The center of the Volt-Var control is an optimization for voltage profiles and power losses governed by networked constraints. Represent a power distribution system as a tree graph (N, ξ), where N is the set of nodes or buses and ξ is the set of edges or lines and transformers.

Since the development of generic language models trained on massive unlabelled text corpora (Radford et al., 2019; Brown et al., 2020), state-of-the art language processing systems rely on sequential transfer learning (Ruder, 2019). The pretrained Language Model (LM) is fine-tuned on the downstream task using a standard supervised learning (SL) objective (Wu et al., 2019; Peters et al., 2019). Yet, such an approach suffers from several issues (Chen et al., 2020): (i) catastrophic forgetting when a model forgets previously learned knowledge and overfits to target domains, (ii) computational inefficiency from fine-tuning billionparameters networks, and (iii) the need of supervised datasets. Moreover, task-specific language models learned with SL suffer from well-studied text degeneration issues (Holtzman et al., 2019), such as the exposure bias (Bengio et al., 2015), language biases (Saleh et al., 2020; Jaques et al., 2020), or a lack of diversity (Li et al., 2015). On the other hand, text generation can be naturally framed as a sequential decision making problem, with the sequence of words seen as successive actions over a vocabulary. Thus, some researchers have recently focused on learning language models using instead Reinforcement Learning (RL) (Strub et al., 2017; Das et al., 2017; Narasimhan et al., 2015).

Awesome Reinforcement Learning - A handpicked collection of Lectures, Books, Surveys, Papers / Thesis, Codes, Tutorials / Websites, Online Demos and Open Source Reinforcement Learning Platforms related to Reinforcement Learning. Machine Learning Cheat Sheets - This repository aims at summing up in the same place all the important notions that are covered in Stanford's CS 229 Machine Learning course, and include: Refreshers in related topics that highlight the key points of the prerequisites of the course and Cheatsheets for each machine learning field, as well as another dedicated to tips and tricks to have in mind when training a model. Deep Learning Papers - This repo covers the most cited papers on various topics like Image Segmentation / Object Detection, Natural Language Processing / RNNs, Reinforcement Learning / Robotics, Convolutional Network Models, Unsupervised / Generative Models, and more. Machine learning Basics - This repository contains implementations of basic machine learning algorithms in plain Python (Python Version 3.6). All algorithms are implemented from scratch without using additional machine learning libraries.

Multi-task learning ideally allows robots to acquire a diverse repertoire of useful skills. However, many multi-task reinforcement learning efforts assume the robot can collect data from all tasks at all times. In reality, the tasks that the robot learns arrive sequentially, depending on the user and the robot's current environment. In this work, we study a practical sequential multi-task RL problem that is motivated by the practical constraints of physical robotic systems, and derive an approach that effectively leverages the data and policies learned for previous tasks to cumulatively grow the robot's skill-set. In a series of simulated robotic manipulation experiments, our approach requires less than half the samples than learning each task from scratch, while avoiding impractical round-robin data collection. On a Franka Emika Panda robot arm, our approach incrementally learns ten challenging tasks, including bottle capping and block insertion.

An unimaginable amount of data is continually being generated by scientific experiments, longitudinal studies, clinical trials, and hospital records--but what can be done with all this information? Barbara Engelhardt (she/her), PhD, is building machine-learning models and statistical tools to make use of that data and find ways to better understand, and even prevent, disease. She is now joining Gladstone Institutes as a senior investigator. "Barbara is an innovator in computational biology," says Katie Pollard, PhD, director of the Gladstone Institute of Data Science and Biotechnology. "She brings vast expertise in statistical models and will help expand our machine-learning program. We're thrilled she's joining our team."