AZoRobotics speaks with Peter Gunnarson from Caltech about his research into using artificial intelligence (AI) to teach autonomous drones to navigate the ocean using ocean currents. An estimated 80% of the ocean is still unexplored, but our current methods for exploring the ocean are often expensive, such as with ship-based observations, or limited in capability, as with buoys. Instead, we are envisioning swarms of smart underwater robots that can navigate on their own, save energy by exploiting ocean currents, find optimal locations to gather data about the ocean and potentially follow and track marine life. This research aims to show how Reinforcement Learning could equip robots with the intelligence needed to accomplish these tasks on their own. With global exploration and sampling of the ocean, we may be able to gain a better understanding of the physics of the ocean and climate change.

Evolutionary strategies have recently been shown to achieve competing levels of performance for complex optimization problems in reinforcement learning. In such problems, one often needs to optimize an objective function subject to a set of constraints, including for instance constraints on the entropy of a policy or to restrict the possible set of actions or states accessible to an agent. Convergence guarantees for evolutionary strategies to optimize stochastic constrained problems are however lacking in the literature. In this work, we address this problem by designing a novel optimization algorithm with a sufficient decrease mechanism that ensures convergence and that is based only on estimates of the functions. We demonstrate the applicability of this algorithm on two types of experiments: i) a control task for maximizing rewards and ii) maximizing rewards subject to a non-relaxable set of constraints.

Originally published on Towards AI the World's Leading AI and Technology News and Media Company. If you are building an AI-related product or service, we invite you to consider becoming an AI sponsor. At Towards AI, we help scale AI and technology startups. Let us help you unleash your technology to the masses. The recommendation systems (RS) are becoming an integral part of our daily lives. This means that we can obtain what we desire either through internet-accessible applications or on social media channels. Traditional views of the recommendation problem refer to it as a simple classification or prediction problem; however, recently new evidence indicates that it is essentially a sequential problem[1]. It can therefore be formulated as a Markov decision process (MDP) and reinforcement learning (RL) methods can be employed to resolve it [1]. RL algorithms play a crucial role as these algorithms are very advantageous to cope with the dynamic environment and large space [4]. Deep Reinforcement Learning (DRL), have enabled RL to be applied to the recommendation problem with massive states and action spaces. RL-based and DRL-based methods in a classified manner based on the specific RL algorithm, like Q-learning, SARSA, and REINFORCE, that is used to optimize the recommendation policy[2].

Cloud computing is a reliable solution to provide distributed computation power. However, real-time response is still challenging regarding the enormous amount of data generated by the IoT devices in 5G and 6G networks. Thus, multi-access edge computing (MEC), which consists of distributing the edge servers in the proximity of end-users to have low latency besides the higher processing power, is increasingly becoming a vital factor for the success of modern applications. This paper addresses the problem of minimizing both, the network delay, which is the main objective of MEC, and the number of edge servers to provide a MEC design with minimum cost. This MEC design consists of edge servers placement and base stations allocation, which makes it a joint combinatorial optimization problem (COP). Recently, reinforcement learning (RL) has shown promising results for COPs. However, modeling real-world problems using RL when the state and action spaces are large still needs investigation. We propose a novel RL framework with an efficient representation and modeling of the state space, action space and the penalty function in the design of the underlying Markov Decision Process (MDP) for solving our problem.

The preferred mode of communication is to email us at distributionalrl@gmail.com.

This is a short guide on how to train an AI to play an arbitrary videogame using reinforcement learning. It shows step-by-step how to set up your custom game environment and train the AI utilizing the Stable-Baselines3 library. I wanted to make this guide accessible, so the presented code is not fully optimized. You can find the source on my GitHub. Unlike its supervised and unsupervised counterparts, Reinforcement Learning (RL) is not about our algorithm learning some underlying truth from a static dataset, instead it interacts with its environment to maximize a reward function (quite similar to how animals are trained in real life with treats).

Efficient credit assignment is essential for reinforcement learning algorithms in both prediction and control settings. We describe a unified view on temporal-difference algorithms for selective credit assignment. These selective algorithms apply weightings to quantify the contribution of learning updates. We present insights into applying weightings to value-based learning and planning algorithms, and describe their role in mediating the backward credit distribution in prediction and control. Within this space, we identify some existing online learning algorithms that can assign credit selectively as special cases, as well as add new algorithms that assign credit backward in time counterfactually, allowing credit to be assigned off-trajectory and off-policy.

From the depths of the oceans to the blackest outposts of space, robots go where we can't. They do the work that's too dangerous or impossible for people, including maintaining infrastructure in hard-to-reach places. In factories, robots help to increase quality and safety on the assembly line. Robots, especially industrial robotic arms, are great candidates for deep reinforcement learning. Deep reinforcement learning (DRL) uses experimentation to train a deep learning solution.

Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable1,2, including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and ‘snowflake’ configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained ‘droplets’ on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied. A newly designed control architecture uses deep reinforcement learning to learn to command the coils of a tokamak, and successfully stabilizes a wide variety of fusion plasma configurations.

Google's DeepMind AI team has collaborated with physicists from the Swiss Plasma Center at EPFL in Ecublens, Switzerland to develop an AI method to control the plasmas inside a nuclear fusion reactor. The study, published in the scientific journal Nature, furthers nuclear fusion research and could help quicken the arrival of a cheaper, clean, unlimited source of energy. DeepMind has now built a neural network using deep reinforcement learning that is able to manipulate the magnetic coils which are essential to confine the soup of plasma at a temperature that is hundreds of millions of degrees Celsius, even hotter than the sun's core. "This AI algorithm, the reinforcement learning, chose to use the TCV coils in a completely different way, which still more or less generates the same magnetic field. So it was still creating the same plasma as we had expected, but it just used the magnetic cores in a completely different way because it had complete freedom to explore the whole operational space. So people were looking at these experimental results about how the coil currents evolve and they were pretty surprised," said EPFL scientist Federico Felici.