Monte-Carlo Planning in Continuous Space MDPs with Non-Asymptotic Analysis ECE and CSL University of Illinois at Urbana-Champaign University of Illinois at Urbana-Champaign Tamer Başar ORIE

Monte-Carlo planning, as exemplified by Monte-Carlo Tree Search (MCTS), has demonstrated remarkable performance in applications with finite spaces. In this paper, we consider Monte-Carlo planning in an environment with continuous state-action spaces, a much less understood problem with important applications in control and robotics.