Deep reinforcement learning with symmetric data augmentation applied for aircraft lateral attitude tracking control

-- Symmetry is an essential property in some dynamical systems that can be exploited for state transition prediction and control policy optimization. This paper develops two symmetry-integrated Reinforcement Learning (RL) algorithms based on standard Deep Deterministic Policy Gradient (DDPG), which leverage environment symmetry to augment explored transition samples of a Markov Decision Process(MDP). The firstly developed algorithm is named as Deep Deterministic Policy Gradient with Symmetric Data Augmentation (DDPG-SDA), which enriches dataset of standard DDPG algorithm by symmetric data augmentation method under symmetry assumption of a dynamical system. T o further improve sample utilization efficiency, the second developed RL algorithm incorporates one extra critic network, which is independently trained with augmented dataset. A two-step approximate policy iteration method is proposed to integrate training for two critic networks and one actor network. The resulting RL algorithm is named as Deep Deterministic Policy Gradient with Symmetric Critic Augmentation (DDPG-SCA). Simulation results demonstrate enhanced sample efficiency and tracking performance of developed two RL algorithms in aircraft lateral tracking control task. I. INTRODUCTION Symmetry property commonly exists in the motions of various mechanical systems, such as aircrafts[1], cars[2] and robotic arms[3]. The common cognition for symmetry property is that the trajectories are symmetric with respect to a reference plane. To be more specific, the knowledge of the trajectory in one symmetric side is predicable according to the knowledge of the trajectory in the other side.