In recent years, there have been many deep structures for Reinforcement Learning, mainly for value function estimation and representations. These methods achieved great success in Atari 2600 domain. In this paper, we propose an improved architecture based upon Dueling Networks, in this architecture, there are two separate estimators, one approximate the state value function and the other, state advantage function. This improvement based on Maximum Entropy, shows better policy evaluation compared to the original network and other value-based architectures in Atari domain.

All the sessions from Transform 2021 are available on-demand now. DeepMind today detailed its latest efforts to create AI systems capable of completing a range of different, unique tasks. By designing a virtual environment called XLand, the Alphabet-backed lab says that it managed to train systems with the ability to succeed at problems and games including hide and seek, capture the flag, and finding objects, some of which they didn't encounter during training. The AI technique known as reinforcement learning has shown remarkable potential, enabling systems to learn to play games like chess, shogi, Go, and StarCraft II through a repetitive process of trial and error. But a lack of training data has been one of the major factors limiting reinforcement learning–trained systems' behavior being general enough to apply across diverse games.

Much work has been dedicated to the exploration of Multi-Agent Reinforcement Learning (MARL) paradigms implementing a centralized learning with decentralized execution (CLDE) approach to achieve human-like collaboration in cooperative tasks. Here, we discuss variations of centralized training and describe a recent survey of algorithmic approaches. The goal is to explore how different implementations of information sharing mechanism in centralized learning may give rise to distinct group coordinated behaviors in multi-agent systems performing cooperative tasks.

Unsupervised skill learning objectives (Gregor et al., 2016, Eysenbach et al., 2018) allow agents to learn rich repertoires of behavior in the absence of extrinsic rewards. They work by simultaneously training a policy to produce distinguishable latent-conditioned trajectories, and a discriminator to evaluate distinguishability by trying to infer latents from trajectories. The hope is for the agent to explore and master the environment by encouraging each skill (latent) to reliably reach different states. However, an inherent exploration problem lingers: when a novel state is actually encountered, the discriminator will necessarily not have seen enough training data to produce accurate and confident skill classifications, leading to low intrinsic reward for the agent and effective penalization of the sort of exploration needed to actually maximize the objective. To combat this inherent pessimism towards exploration, we derive an information gain auxiliary objective that involves training an ensemble of discriminators and rewarding the policy for their disagreement. Our objective directly estimates the epistemic uncertainty that comes from the discriminator not having seen enough training examples, thus providing an intrinsic reward more tailored to the true objective compared to pseudocount-based methods (Burda et al., 2019). We call this exploration bonus discriminator disagreement intrinsic reward, or DISDAIN. We demonstrate empirically that DISDAIN improves skill learning both in a tabular grid world (Four Rooms) and the 57 games of the Atari Suite (from pixels). Thus, we encourage researchers to treat pessimism with DISDAIN.

Swarm optimization algorithms are widely used for feature selection before data mining and machine learning applications. The metaheuristic nature-inspired feature selection approaches are used for single-objective optimization tasks, though the major problem is their frequent premature convergence, leading to weak contribution to data mining. In this paper, we propose a novel feature selection algorithm named Reinforced Swarm Optimization (RSO) leveraging some of the existing problems in feature selection. This algorithm embeds the widely used Bee Swarm Optimization (BSO) algorithm along with Reinforcement Learning (RL) to maximize the reward of a superior search agent and punish the inferior ones. This hybrid optimization algorithm is more adaptive and robust with a good balance between exploitation and exploration of the search space. The proposed method is evaluated on 25 widely known UCI datasets containing a perfect blend of balanced and imbalanced data. The obtained results are compared with several other popular and recent feature selection algorithms with similar classifier configurations. The experimental outcome shows that our proposed model outperforms BSO in 22 out of 25 instances (88%). Moreover, experimental results also show that RSO performs the best among all the methods compared in this paper in 19 out of 25 cases (76%), establishing the superiority of our proposed method.

Reinforcement learning (RL) has shown a promising performance in learning optimal policies for a variety of sequential decision-making tasks. However, in many real-world RL problems, besides optimizing the main objectives, the agent is expected to satisfy a certain level of safety (e.g., avoiding collisions in autonomous driving). While RL problems are commonly formalized as Markov decision processes (MDPs), safety constraints are incorporated via constrained Markov decision processes (CMDPs). Although recent advances in safe RL have enabled learning safe policies in CMDPs, these safety requirements should be satisfied during both training and in the deployment process. Furthermore, it is shown that in memory-based and partially observable environments, these methods fail to maintain safety over unseen out-of-distribution observations. To address these limitations, we propose a Lyapunov-based uncertainty-aware safe RL model. The introduced model adopts a Lyapunov function that converts trajectory-based constraints to a set of local linear constraints. Furthermore, to ensure the safety of the agent in highly uncertain environments, an uncertainty quantification method is developed that enables identifying risk-averse actions through estimating the probability of constraint violations. Moreover, a Transformers model is integrated to provide the agent with memory to process long time horizons of information via the self-attention mechanism. The proposed model is evaluated in grid-world navigation tasks where safety is defined as avoiding static and dynamic obstacles in fully and partially observable environments. The results of these experiments show a significant improvement in the performance of the agent both in achieving optimality and satisfying safety constraints.

Cassie has made history as the first bipedal robot to complete a five-kilometer (5K) run, having done so in just over 53 minutes. Developed by Oregon State University, the two-legged machine with knees that bend like those of an ostrich, taught itself how to run through a deep reinforcement learning algorithm. Yesh Godse, an undergraduate in the lab, said in a statement: 'Deep reinforcement learning is a powerful method in AI that opens up skills like running, skipping and walking up and down stairs.' Cassie's total time of 53 minutes, three seconds, included about six and a half minutes of resets following two falls. Cassie first stumbled when its computer overheated and the other came after it took a turn at too high of a speed. The robot's makers foresee it eventually delivering packages, managing warehouse tasks and helping people in their homes.

Introduction to Reinforcement Learning

Cassie, a bipedal robot that's all legs, has successfully ran five kilometers without having a tether and on a single charge. The machine serves as the basis for Agility Robotics' delivery robot Digit, as TechCrunch notes, though you may also remember it for "blindly" navigating a set of stairs. Oregon State University engineers were able to train Cassie in a simulator to give it the capability to go up and down a flight of stairs without the use of cameras or LIDAR. Now, engineers from the same team were able to train Cassie to run using a deep reinforcement learning algorithm. According to the team, Cassie teaching itself using the technique gave it the capability to stay upright without a tether by shifting its balance while running.

Four years is a long time in robotics, especially so for a bipedal robot developed at Oregon State University (OSU) named Cassie. Dreamt up as an agile machine to carry packages from delivery vans to doorsteps, Cassie has recently developed an ability to run, something its developers have now shown off by having it complete what they say is the first 5-km (3.1-mi) jog by a bipedal robot. We first took a look at Cassie the bipedal robot back in 2017, when OSU researchers revealed an ostrich-like machine capable of waddling along at a steady pace. It is based on the team's previously developed Atrias bipedal robot, but featured steering feet and sealed electronics in order to function in the rain and snow and navigate outdoor terrain. The team has since used machine learning to equip Cassie with an impressive new skill: the ability to run. This involved what they call a deep reinforcement learning algorithm, which Cassie combines with its unique biomechanics and knees that bend like an ostrich to make fine adjustments to keep itself upright when on the move.