SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning

Much of the success of modern technology can be attributed to our ability to control dynamical systems: designing safe biomedical implants for homeostatic regulation, gimbling rocket boosters for reusable launch vehicles, operating power plants and power grids, industrial manufacturing, among many other examples. Over the past decade, advances in machine learning and optimization have rapidly accelerated our capabilities to tackle complicated data-driven tasks--particularly in the fields of computer vision [1] and natural language processing [2]. Reinforcement learning (RL) is at the intersection of both machine learning and optimal control, and the core ideas of RL date back to the infancy of both fields. By interacting with an environment and receiving feedback about its performance on a task through a reward, an RL agent iteratively improves a control policy. Deep reinforcement learning (DRL), in particular, has shown promise for uncovering control policies in complex, high-dimensional spaces [3-11]. DRL has been used to achieve super-human performance in games [12-16] and drone racing [17], to control the plasma dynamics in a tokamak fusion reactor [18], to discover novel drugs [19], and for many applications in fluid mechanics [20-30]. However, these methods rely on neural networks and typically suffer from three major drawbacks: (1) they are infeasible to train for many applications because they require millions-- or even billions [16]--of interactions with the environment; (2) they are challenging to deploy in resource-constrained environments (such as embedded devices and micro-robotic systems) due to the size of the networks and need for specialized software; and (3) they are "black-box" models that lack interpretability, making them untrustworthy to operate in safety-critical systems or high-consequence environments. In this work, we seek to create interpretable and generalizable reinforcement learning methods that are also more sample efficient via sparse dictionary learning.