We propose using regularization for Multi-Agent Reinforcement Learning rather than learning explicit cooperative structures called {\em Multi-Agent Regularized Q-learning} (MARQ). Many MARL approaches leverage centralized structures in order to exploit global state information or removing communication constraints when the agents act in a decentralized manner. Instead of learning redundant structures which is removed during agent execution, we propose instead to leverage shared experiences of the agents to regularize the individual policies in order to promote structured exploration. We examine several different approaches to how MARQ can either explicitly or implicitly regularize our policies in a multi-agent setting. MARQ aims to address these limitations in the MARL context through applying regularization constraints which can correct bias in off-policy out-of-distribution agent experiences and promote diverse exploration. Our algorithm is evaluated on several benchmark multi-agent environments and we show that MARQ consistently outperforms several baselines and state-of-the-art algorithms; learning in fewer steps and converging to higher returns.

Reinforcement learning has been shown to perform a range of complex tasks through interaction with an environment or collected leveraging experience. However, many of these approaches presume optimal or near optimal experiences or the presence of a consistent environment. In this work we propose dual, advantage-based behavior policy based on counterfactual regret minimization. We demonstrate the flexibility of this approach and how it can be adapted to online contexts where the environment is available to collect experiences and a variety of other contexts. We demonstrate this new algorithm can outperform several strong baseline models in different contexts based on a range of continuous environments. Additional ablations provide insights into how our dual behavior regularized reinforcement learning approach is designed compared with other plausible modifications and demonstrates its ability to generalize.

This paper introduces Greedy UnMix (GUM) for cooperative multi-agent reinforcement learning (MARL). Greedy UnMix aims to avoid scenarios where MARL methods fail due to overestimation of values as part of the large joint state-action space. It aims to address this through a conservative Q-learning approach through restricting the state-marginal in the dataset to avoid unobserved joint state action spaces, whilst concurrently attempting to unmix or simplify the problem space under the centralized training with decentralized execution paradigm. We demonstrate the adherence to Q-function lower bounds in the Q-learning for MARL scenarios, and demonstrate superior performance to existing Q-learning MARL approaches as well as more general MARL algorithms over a set of benchmark MARL tasks, despite its relative simplicity compared with state-of-the-art approaches.

In this article, we summarise the content from our focus series on the UN Sustainable Development Goal (SDG) number 14: life below water, and we highlight further interesting research in the field. The UN write that the aim of this goal is to: "Conserve and sustainably use the oceans, seas and marine resources for sustainable development." This includes topics such as reducing marine pollution, protecting and restoring ecosystems, reducing ocean acidification, and sustainable fishing. The aim of the OcéanIA project is to develop new artificial intelligence and mathematical modelling tools to contribute to the understanding of the oceans and their role in regulating and sustaining the biosphere, and tackling climate change. We interviewed Nayat Sánchez-Pi, Director of the Inria Chile Research Center, who told us more about this important and exciting project.

Australia on Friday shrugged off Chinese anger over its decision to acquire US nuclear-powered submarines, while vowing to defend the rule of law in airspace and waters where Beijing has staked hotly contested claims. US President Joe Biden announced the new Australia-US-Britain defence alliance on Wednesday, extending US nuclear submarine technology to Australia as well as cyber defence, applied artificial intelligence and undersea capabilities. Beijing described the new alliance as an "extremely irresponsible" threat to regional stability, questioning Australia's commitment to nuclear non-proliferation and warning the Western allies that they risked "shooting themselves in the foot". China has its own "very substantive programme of nuclear submarine building", Australian Prime Minister Scott Morrison argued Friday in an interview with radio station 2GB. "They have every right to take decisions in their national interests for their defence arrangements and of course so does Australia and all other countries," he said.

USA and China are turning out be a landscape of the biggest markets in the present scenario. Taking a lead over the world is globally at power to grow and evolve in this new era in the economic sector. Artificial Intelligence field is a big enterprise in the present time. Among the economic giants of the world, the USA and China have taken over the journey. The navigation is still on by the early adopters and more familiar one's towards exploring the further transformation which can take in this field.

Australia on Friday shrugged off Chinese anger over its decision to acquire US nuclear-powered submarines and vowed to defend the rule of law in airspace and waters where Beijing has staked multiple hotly contested claims. US President Joe Biden announced the new Australia-US-Britain defence alliance on Wednesday, extending US nuclear submarine technology to Australia as well as cyber defence, applied artificial intelligence and undersea capabilities. China's government described the alliance as an "extremely irresponsible" threat to regional stability, questioning Australia's commitment to nuclear non-proliferation and warning the Western allies that they risked "shooting themselves in the foot". China has its own "very substantive programme of nuclear submarine building", Australian Prime Minister Scott Morrison said Friday in an interview with radio station 2GB. "They have every right to take decisions in their national interests for their defence arrangements and of course so does Australia and all other countries," he said.

This paper looks at solving collaborative planning problems formalized as Decentralized POMDPs (Dec-POMDPs) by searching for Nash equilibria, i.e., situations where each agent's policy is a best response to the other agents' (fixed) policies. While the Joint Equilibrium-based Search for Policies (JESP) algorithm does this in the finite-horizon setting relying on policy trees, we propose here to adapt it to infinite-horizon Dec-POMDPs by using finite state controller (FSC) policy representations. In this article, we (1) explain how to turn a Dec-POMDP with $N-1$ fixed FSCs into an infinite-horizon POMDP whose solution is an $N^\text{th}$ agent best response; (2) propose a JESP variant, called \infJESP, using this to solve infinite-horizon Dec-POMDPs; (3) introduce heuristic initializations for JESP aiming at leading to good solutions; and (4) conduct experiments on state-of-the-art benchmark problems to evaluate our approach.

Future manufacturing requires complex systems that connect simulation platforms and virtualization with physical data from industrial processes. Digital twins incorporate a physical twin, a digital twin, and the connection between the two. Benefits of using digital twins, especially in manufacturing, are abundant as they can increase efficiency across an entire manufacturing life-cycle. The digital twin concept has become increasingly sophisticated and capable over time, enabled by rises in many technologies. In this paper, we detail the cognitive digital twin as the next stage of advancement of a digital twin that will help realize the vision of Industry 4.0. Cognitive digital twins will allow enterprises to creatively, effectively, and efficiently exploit implicit knowledge drawn from the experience of existing manufacturing systems. They also enable more autonomous decisions and control, while improving the performance across the enterprise (at scale). This paper presents graph learning as one potential pathway towards enabling cognitive functionalities in manufacturing digital twins. A novel approach to realize cognitive digital twins in the product design stage of manufacturing that utilizes graph learning is presented.

When starting out to deploy artificial intelligence (AI) and machine learning (ML), executives of legacy companies often view the challenges mainly as technical problems -- particularly finding sources of internal data to analyze and choosing the right tools. What they may not appreciate is just how data-rich their legacy companies already are. From utilities and mining, transportation and shipping, to financial services and more, legacy company operations and customer interactions generate a wealth of data. Such data can be harnessed to tackle a very wide range of issues: optimizing supply chains, predicting maintenance, reducing accidents, increasing production output, improving operational efficiency, raising revenue productivity, and growing customer value. To realize these opportunities using AI, however, legacy companies worldwide typically soon discover that their biggest problem is not technology -- it's talent.