Despite a series of recent successes in reinforcement learning (RL), many RL algorithms remain sensitive to hyperparameters. As such, there has recently been interest in the field of AutoRL, which seeks to automate design decisions to create more general algorithms. Recent work suggests that population based approaches may be effective AutoRL algorithms, by learning hyperparameter schedules on the fly. In particular, the PB2 algorithm is able to achieve strong performance in RL tasks by formulating online hyperparameter optimization as time varying GP-bandit problem, while also providing theoretical guarantees. However, PB2 is only designed to work for \emph{continuous} hyperparameters, which severely limits its utility in practice. In this paper we introduce a new (provably) efficient hierarchical approach for optimizing \emph{both continuous and categorical} variables, using a new time-varying bandit algorithm specifically designed for the population based training regime. We evaluate our approach on the challenging Procgen benchmark, where we show that explicitly modelling dependence between data augmentation and other hyperparameters improves generalization.

People Can Fly is one of the leading independent AAA games development studios with an international team of hundreds of talented individuals working from offices located in Poland, UK, US, and Canada, and from all over the world thanks to our remote work programs. Founded in 2002, we made our mark on the shooter genre with titles such as Painkiller, Bulletstorm, Gears of War: Judgment, and Outriders. We are one of the most experienced Unreal Engine studios in the industry and we are expanding it with in-house solutions called PCF Framework. Our creative teams are currently working on several exciting titles: Gemini is our new project being developed with Square Enix; Bifrost, Victoria and Dagger are projects we're growing in the self-publishing model. We also have one project in the concept phase – Red; as well as two projects in VR technology – Green Hell VR and Thunder - a new project based on one of the IPs from the Group's portfolio.

People Can Fly is one of the leading independent AAA games development studios with an international team of hundreds of talented individuals working from offices located in Poland, UK, US, and Canada, and from all over the world thanks to our remote work programs. Founded in 2002, we made our mark on the shooter genre with titles such as Painkiller, Bulletstorm, Gears of War: Judgment, and Outriders. We are one of the most experienced Unreal Engine studios in the industry and we are expanding it with in-house solutions called PCF Framework. Our creative teams are currently working on several exciting titles: Gemini is our new project being developed with Square Enix; Bifrost, Victoria and Dagger are projects we're growing in the self-publishing model. We also have one project in the concept phase – Red; as well as two projects in VR technology – Green Hell VR and Thunder - a new project based on one of the IPs from the Group's portfolio.

Planning--the ability to synthesize a course of action to achieve desired goals--is an important part of intelligent agency and has thus received significant attention within AI for more than 30 years. Work on efficient planning algorithms still continues to be a hot topic for research in AI and has led to several exciting developments in the past few years. This article provides a tutorial introduction to all the algorithms and approaches to the planning problem in AI. To fulfill this ambitious objective, I introduce a generalized approach to plan synthesis called refinement planning and show that in its various guises, refinement planning subsumes most of the algorithms that have been, or are being, developed. It is hoped that this unifying overview provides the reader with a brand-name-free appreciation of the essential issues in planning.

"Learning to Fly by Crashing," a paper from CMU roboticists Dhiraj Gandhi, Lerrel Pinto, and Abhinav Gupta, has such a nice abstract that I'll just let them explain what this research is all about: The reason most research avoids using large-scale real data is the fear of crashes! In this paper, we propose to bite the bullet and collect a dataset of crashes itself! We build a drone whose sole purpose is to crash into objects [. . We use all this negative flying data in conjunction with positive data sampled from the same trajectories to learn a simple yet powerful policy for UAV navigation. One way to think of flying (or driving or walking or any other form of motion) is that success is simply a continual failure to crash.

We present a framework for reasoning about actions that not only solves the frame and ramification problems, but also the state default problem—the problem to determine what normally holds at a given time point. Yet, the framework is general enough not to be tied to a specific time structure. This is achieved as follows: We use effect axioms that draw ideas both from Reiter's successor state axioms and the non-monotonic causal theories by Giunchiglia et al. These axioms are formulated in a recently proposed unifying action calculus to guarantee independence of a specific underlying notion of time. Reiter's default logic is then wrapped around the resulting calculus and plays a key role in solving the ramification as well as the state default problem.

Planning -- the ability to synthesize a course of action to achieve desired goals -- is an important part of intelligent agency and has thus received significant attention within AI for more than 30 years. Work on efficient planning algorithms still continues to be a hot topic for research in AI and has led to several exciting developments i the past few years. This article provides a tutorial introduction to all the algorithms and approaches to the planning problem in AI. To fulfill this ambitious objective, I introduce a generalized approach to plan synthesis called refinement planning and show that in its various guises, refinement planning subsumes most of the algorithms that have been, or are being, developed. It is hoped that this unifying overview provides the reader with a brand-name-free appreciation of the essential issues in planning.