In a warehouse environment, tasks appear dynamically. Consequently, a task management system that matches them with the workforce too early (e.g., weeks in advance) is necessarily sub-optimal. Also, the rapidly increasing size of the action space of such a system consists of a significant problem for traditional schedulers. Reinforcement learning, however, is suited to deal with issues requiring making sequential decisions towards a long-term, often remote, goal. In this work, we set ourselves on a problem that presents itself with a hierarchical structure: the task-scheduling, by a centralised agent, in a dynamic warehouse multi-agent environment and the execution of one such schedule, by decentralised agents with only partial observability thereof. We propose to use deep reinforcement learning to solve both the high-level scheduling problem and the low-level multi-agent problem of schedule execution. Finally, we also conceive the case where centralisation is impossible at test time and workers must learn how to cooperate in executing the tasks in an environment with no schedule and only partial observability.

We introduce a method for pricing consumer credit using recent advances in offline deep reinforcement learning. This approach relies on a static dataset and requires no assumptions on the functional form of demand. Using both real and synthetic data on consumer credit applications, we demonstrate that our approach using the conservative Q-Learning algorithm is capable of learning an effective personalized pricing policy without any online interaction or price experimentation.

In this paper, we study gap-dependent regret guarantees for risk-sensitive reinforcement learning based on the entropic risk measure. We propose a novel definition of sub-optimality gaps, which we call cascaded gaps, and we discuss their key components that adapt to the underlying structures of the problem. Based on the cascaded gaps, we derive non-asymptotic and logarithmic regret bounds for two model-free algorithms under episodic Markov decision processes. We show that, in appropriate settings, these bounds feature exponential improvement over existing ones that are independent of gaps. We also prove gap-dependent lower bounds, which certify the near optimality of the upper bounds.

The combination of deep learning and decision learning has led to several impressive stories in decision-making AI research, including AIs that can play a variety of games (Atari video games, board games, complex real-time strategy game Starcraft II), control robots (in simulation and in the real world), and even fly a weather balloon. These are examples of sequential decision tasks, in which the AI agent needs to make a sequence of decisions to achieve its goal. Today, the two main approaches for training such agents are reinforcement learning (RL) and imitation learning (IL). In reinforcement learning, humans provide rewards for completing discrete tasks, with the rewards typically being delayed and sparse. For example, 100 points are given for solving the first room of Montezuma's revenge (Fig.1). In the imitation learning setting, humans can transfer knowledge and skills through step-by-step action demonstrations (Fig.2), and the agent then learns to mimic human actions.

Designing a distribution of environments in which RL agents can learn interesting and useful skills is a challenging and poorly understood task, for multi-agent environments the difficulties are only exacerbated. One approach is to train a second RL agent, called a teacher, who samples environments that are conducive for the learning of student agents. However, most previous proposals for teacher rewards do not generalize straightforwardly to the multi-agent setting. We examine a set of intrinsic teacher rewards derived from prediction problems that can be applied in multi-agent settings and evaluate them in Mujoco tasks such as multiagent Hide and Seek [1] as well as a diagnostic single-agent maze task. Of the intrinsic rewards considered we found value disagreement to be most consistent across tasks, leading to faster and more reliable emergence of advanced skills in Hide and Seek and the maze task. Another candidate intrinsic reward considered, value prediction error, also worked well in Hide and Seek but was susceptible to noisy-TV style distractions in stochastic environments. Policy disagreement performed well in the maze task but did not speed up learning in Hide and Seek. Our results suggest that intrinsic teacher rewards, and in particular value disagreement, are a promising approach for automating both single and multi-agent environment design.

Every two years, the current AAAI president gives the opening address at the AAAI Conference on Artificial Intelligence. This year it was the turn of Bart Selman. In his talk he reviewed the current state of AI and presented examples of three different applications of AI to aid scientific discovery. Bart began his talk by considering the deep-learning "revolution", highlighting some of the areas that it has transformed, namely computer vision, natural language processing, machine translation, game play, and reinforcement learning. He noted that the field is undergoing a rapid acceleration at the moment, with an incredible rate of progress.

The RL environment maintains the states of the grounded logistics planning domain. In the experiment, we define a planning task as an abstract planning task that is defined over the subset of the predicates and actions in the grounded logistics planning task. Therefore, the state mapping function is a projection of logical variables from an RL state to its planning state. The PDDL domain file used for the RL task is desribed as follows.

Reinforcement learning has had a great impact in decision making problems, in particular combined with artificial neural networks [1, 2]. Nevertheless, alternatives to neural networks are still needed for a number of different reasons, first, because the amount of data is expected to continue to grow along with its dimensionality, and, second, neural networks carry vulnerabilities that make them prone to adversarial attacks [3]. A possible alternative to deep learning for further improving machine learning can be found in quantum computing that has shown to be able to perform tasks beyond the reach of classical computing [4]. The field of quantum machine learning explores how to design and implement quantum algorithms that could enable machine learning that is faster, more expressive, or more explainable. Using quantum computers, a number of quantum machine learning algorithms have been published for supervised and unsupervised learning [5-11].

Consequently, schemes for numerical discretization are a key element of scientific computing, and one of the main challenges is to determine a good trade-off between the required accuracy and numerical efficiency. While the standard approach to developing quadrature rules and numerical schemes to solve ODEs is based on Taylor series expansions and the associated error bounds determined by higher-order derivatives [2], the advances in data science and machine learning have recently fueled the development of alternative concepts that are based on training data. Most of these approaches are developed with the aim to efficiently compute the numerical solution of dynamical systems of high complexity, see, for instance, [18, 25, 15, 19, 20, 12], where the flow map F that takes a state x at time t to a future state x(t + t) is approximated. In contrast to that, our work addresses the task of efficiently performing numerical integration for integrands or differential equations of a given problem class to a desired accuracy. To this end, the next sample point at which the integrand or force term is evaluated is determined by finding an optimal trade-off between the two conflicting criteria accuracy and numerical efficiency. This task is carried out by a reinforcement learning algorithm which, taking past function evaluations and learned knowledge about the problem class into account, determines the next sample point at which to evaluate the function of interest. The efficiency of the proposed approach in comparison to state-of-the-art methods will be demonstrated using examples from the area of computing integrals (quadrature) as well as numerically solving differential equations.

This thesis rigorously studies fundamental reinforcement learning (RL) methods in modern practical considerations, including robust RL, distributional RL, and offline RL with neural function approximation. The thesis first prepares the readers with an overall overview of RL and key technical background in statistics and optimization. In each of the settings, the thesis motivates the problems to be studied, reviews the current literature, provides computationally efficient algorithms with provable efficiency guarantees, and concludes with future research directions. The thesis makes fundamental contributions to the three settings above, both algorithmically, theoretically, and empirically, while staying relevant to practical considerations.