Reinforcement learning of optimal active particle navigation

The development of self-propelled particles at the micro-and the nanoscale has sparked a huge potential for future applications in active matter physics, microsurgery, and targeted drug delivery. However, while the latter applications provoke the quest on how to optimally navigate towards a target, such as e.g. a cancer cell, there is still no simple way known to determine the optimal route in sufficiently complex environments. Here we develop a machine learning-based approach that allows us, for the first time, to determine the asymptotically optimal path of a selfpropelled agent which can freely steer in complex environments. Our method hinges on policy gradient-based deep reinforcement learning techniques and, crucially, does not require any reward shaping or heuristics. The presented method provides a powerful alternative to current analytical methods to calculate optimal trajectories and opens a route towards a universal path planner for future intelligent active particles. Keywords: Active Matter Physics, Colloids, Soft Matter Physics, Optimization, Microswimmers, Optimal Navigation, Reinforcement Learning Introduction The problem of finding suitable navigation strategies is of great interest to applications ranging from motion planning for autonomous underwater vehicles, ocean gliders [1-4], and aerial vehicles [5-7] to microorganisms searching for food and prey [8,9] and striving for survival in complex environments [10, 11]. One important class of path planning problems which is currently attracting a rapidly increasing attention is centered around the quest for the optimal trajectory allowing an active particle, which can freely steer but cannot control its speed, to reach a given target in a complex environment. This active particle navigation (APN) problem is relevant both for biological swimmers like fish or for turtles on the way to their breeding grounds [12,13] and for future applications of synthetic microswimmers [14] such as targeted drug [15-17] and gene delivery [18,19] or microsurgery [20].