The baselines are somehow weak. Though TorchBeast is a strong baseline, the PPO and A2C from Kostrikov seem weak. As far as I know, faster training is not the goal of Kostrikov's implementation. For PPO, the implementation from OpenAI baselines are stronger, which features parallelization with MPI and all-reduce gradients. For A2C, one could consider rlpyt (rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch), where various sampling schemes (including batch synchronization) and optimization schemes can be used.

Researchers from all over the world contribute to this repository as a prelude to the peer review process for publication in traditional journals. We hope to save you some time by picking out articles that represent the most promise for the typical data scientist. The articles listed below represent a fraction of all articles appearing on the preprint server. They are listed in no particular order with a link to each paper along with a brief overview. Especially relevant articles are marked with a "thumbs up" icon.

Boston Dynamics' Robot Dog Is Now Available for Select Customers Boston Dynamics has begun commercialization of its robodog Spot. The company released a video on Tuesday that shows Spot navigating challenging terrain, picking up construction objects, moving through bad weather, and picking itself up after a fall. Boston Dynamics' Atlas Can Now Do An Impressive Gymnastics Routine Alongside the news that Boston Dynamics is letting robot dog Spot out of its laboratory for the first time, the company has released a new video of Atlas, a spectacular bipedal robot that's previously been seen doing everything from parkour to backflips. Contributing Data to Deepfake Detection Research In collaboration with Jigsaw, Google has announced the release of a large dataset of visual deepfakes they have produced. The data has been incorporated into the Technical University of Munich and the University Federico II of Naples' new FaceForensics benchmark, an effort that Google co-sponsors.

Since the recent advent of deep reinforcement learning for game play and simulated robotic control, a multitude of new algorithms have flourished. Most are model-free algorithms which can be categorized into three families: deep Q-learning, policy gradients, and Q-value policy gradients. These have developed along separate lines of research, such that few, if any, code bases incorporate all three kinds. Yet these algorithms share a great depth of common deep reinforcement learning machinery. We are pleased to share rlpyt, which implements all three algorithm families on top of a shared, optimized infrastructure, in a single repository. It contains modular implementations of many common deep RL algorithms in Python using PyTorch, a leading deep learning library. rlpyt is designed as a high-throughput code base for small- to medium-scale research in deep RL. This white paper summarizes its features, algorithms implemented, and relation to prior work, and concludes with detailed implementation and usage notes. rlpyt is available at https://github.com/astooke/rlpyt.