Multistep returns, such as $n$-step returns and $\lambda$-returns, are commonly used to improve the sample efficiency of reinforcement learning (RL) methods. The variance of the multistep returns becomes the limiting factor in their length; looking too far into the future increases variance and reverses the benefits of multistep learning. In our work, we demonstrate the ability of compound returns -- weighted averages of $n$-step returns -- to reduce variance. We prove for the first time that any compound return with the same contraction modulus as a given $n$-step return has strictly lower variance. We additionally prove that this variance-reduction property improves the finite-sample complexity of temporal-difference learning under linear function approximation. Because general compound returns can be expensive to implement, we introduce two-bootstrap returns which reduce variance while remaining efficient, even when using minibatched experience replay. We conduct experiments showing that two-bootstrap returns can improve the sample efficiency of $n$-step deep RL agents, with little additional computational cost.

Artificial intelligence is upending the financial management industry in spectacular ways. The majority of machine learning and deep learning solutions have focused on fundamental analysis of securities. However, deep learning and other artificial intelligence technologies will also change the future of technical analysis as well. A number of experts have started analyzing the role of AI in technical analysis. One white paper published on Science Direct shows that it could be one of the biggest breakthroughs in modern financial trading.

Artificial intelligence is upending the financial management industry in spectacular ways. The majority of machine learning and deep learning solutions have focused on fundamental analysis of securities. However, deep learning and other artificial intelligence technologies will also change the future of technical analysis as well. A number of experts have started analyzing the role of AI in technical analysis. One white paper published on Science Direct shows that it could be one of the biggest breakthroughs in modern financial trading.