Using Simulation Optimization to Improve Zero-shot Policy Transfer of Quadrotors

Programming intelligent control strategies for complex robot systems is a challenging task. Reinforcement learning (RL) promises the automated synthesis of control strategies through a data-driven approach instead of explicitly designing hand-crafted solutions through expert knowledge. In recent years, the field of RL has witnessed outstanding successes and raised a surge of interest in the control of dynamical systems through such a trial-and-error paradigm. The combination of RL and deep learning methods excel at problems that can be quickly simulated like robotics [13, 22] or video games [18, 29] and in domains where the exact model is known but long-horizon planning is not computationally tractable, e.g. board games like Go and Chess [26]. Despite the significant advances in recent years, the applicability of RL algorithms is still limited when the data at test time differ from those seen during training [10]. Since many real-world systems cannot afford to learn policies from scratch due to the expense of data, simulations are the preferred approach to build data-driven control policies in the RL community. In fact, a gap between the simulation and the real world still persists because the modeling of all effects either requires in-depth expert knowledge or is simply not desirable, e.g.