Deep Reinforcement Learning in Portfolio Management

Abstract--In this paper, we implement two state-of-art continuous reinforcement learning algorithms, Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO) in portfolio management. Both of them are widely-used in game playing and robot control. What's more, PPO has appealing theoretical propeties which is hopefully potential in portfolio management. We present the performances of them under different settings, including different learning rate, objective function, markets, feature combinations, in order to provide insights for parameter tuning, features selection and data preparation. Utilizing deep reinforcement learning in portfolio management is gaining popularity in the area of algorithmic trading. However, deep learning is notorious for its sensitivity to neural network structure, feature engineering and so on. Therefore, in our experiments, we explored influences of different optimizers and network structures on trading agents utilizing two kinds of deep reinforcement learning algorithms, deep deterministic policy gradient (DDPG) and proximal policy optimization (PPO). Our experiments were conveyed on datasets of China and America stock market.