Adaptive Guidance with Reinforcement Meta-Learning

This paper proposes a novel adaptive guidance system developed using reinforcement meta-learning with a recurrent policy and value function approximator. The use of recurrent network layers allows the deployed policy to adapt real time to environmental forces acting on the agent. We compare the performance of the DR/DV guidance law, an RL agent with a non-recurrent policy, and an RL agent with a recurrent policy in four difficult tasks with unknown but highly variable dynamics. These tasks include a safe Mars landing with random engine failure and a landing on an asteroid with unknown environmental dynamics. We also demonstrate the ability of a recurrent policy to navigate using only Doppler radar altimeter returns, thus integrating guidance and navigation. INTRODUCTION Many space missions take place in environments with complex and time-varying dynamics that may be incompletely modeled during the mission design phase. For example, during an orbital refueling mission, the inertia tensor of each of the two spacecraft will change significantly as fuel is transferred from one spacecraft to the other, which can make the combined system difficult to control. The wet mass of an exoatmospheric kill vehicles (EKV) consists largely of fuel, and as this is depleted with divert thrusts, the center of mass changes, and the divert thrusts are no longer orthogonal to the EKV's velocity vector, which wastes fuel and impacts performance. Future missions to asteroids might be undertaken before the asteroid's gravitational field, rotational velocity, and local solar radiation pressure are accurately modeled.