The importance of hyperparameter optimization for model-based reinforcement learning

Model-based reinforcement learning (MBRL) is a variant of the iterative learning framework, reinforcement learning, that includes a structured component of the system that is solely optimized to model the environment dynamics. Learning a model is broadly motivated from biology, optimal control, and more – it is grounded in natural human intuition of planning before acting. In this post, we discuss how model-based reinforcement learning is more susceptible to parameter tuning and how AutoML can help in finding very well performing parameter settings and schedules. Below, the top animation is the expected behavior of an agent maximizing velocity on a "Half Cheetah" robotic task, and underneath is what our paper with hyperparameter tuning finds. Model-based reinforcement learning (MBRL) is an iterative framework for solving tasks in a partially understood environment.