Statistically Efficient Advantage Learning for Offline Reinforcement Learning in Infinite Horizons

Reinforcement learning (RL, see Sutton and Barto, 2018, for an overview) is concerned with how intelligence agents learn and take actions in an unknown environment in order to maximize the cumulative reward that it receives. It has been arguably one of the most vibrant research frontiers in machine learning over the last few years. According to Google Scholar, over 40K scientific articles have been published in 2020 with the phrase "reinforcement learning". Over 100 papers on RL were accepted for presentation at ICML 2021, a premier conference in the machine learning area, accounting for more than 10% of the accepted papers in total. RL algorithms have been applied in a wide variety of real applications, including games (Silver et al., 2016), robotics (Kormushev et al., 2013), healthcare (Komorowski et al., 2018), bidding (Jin et al., 2018), ridesharing (Xu et al., 2018) and automated driving (de Haan et al., 2019), to name a few. This paper is partly motivated by developing statistical learning methodologies in offline RL domains such as mobile health (mHealth).