In the field of cooperative multi-agent reinforcement learning (MARL), the standard paradigm is the use of centralised training and decentralised execution where a central critic conditions the policies of the cooperative agents based on a central state. It has been shown, that in cases with large numbers of redundant agents these methods become less effective. In a more general case, there is likely to be a larger number of agents in an environment than is required to solve the task. These redundant agents reduce performance by enlarging the dimensionality of both the state space and and increasing the size of the joint policy used to solve the environment. We propose leveraging layerwise relevance propagation (LRP) to instead separate the learning of the joint value function and generation of local reward signals and create a new MARL algorithm: relevance decomposition network (RDN). We find that although the performance of both baselines VDN and Qmix degrades with the number of redundant agents, RDN is unaffected.

The FC Portugal 3D team is developed upon the structure of our previous Simulation league 2D/3D teams and our standard platform league team. Our research concerning the robot low-level skills is focused on developing behaviors that may be applied on real robots with minimal adaptation using model-based approaches. Our research on high-level soccer coordination methodologies and team playing is mainly focused on the adaptation of previously developed methodologies from our 2D soccer teams to the 3D humanoid environment and on creating new coordination methodologies based on the previously developed ones. The research-oriented development of our team has been pushing it to be one of the most competitive over the years (World champion in 2000 and Coach Champion in 2002, European champion in 2000 and 2001, Coach 2nd place in 2003 and 2004, European champion in Rescue Simulation and Simulation 3D in 2006, World Champion in Simulation 3D in Bremen 2006 and European champion in 2007, 2012, 2013, 2014 and 2015). This paper describes some of the main innovations of our 3D simulation league team during the last years. A new generic framework for reinforcement learning tasks has also been developed. The current research is focused on improving the above-mentioned framework by developing new learning algorithms to optimize low-level skills, such as running and sprinting. We are also trying to increase student contact by providing reinforcement learning assignments to be completed using our new framework, which exposes a simple interface without sharing low-level implementation details.

A smart home can be considered a place of residence that enables the management of appliances and systems to help with day-to-day life by automated technology. In the current paper is described a prototype that simulates a contextaware environment, developed in a designed smart home. The smart home environment has been simulated using three agents and five locations in a house. The context-aware agents behave based on predefined rules designed for daily activities. Our proposal aims to reduce operational cost of running devices. In the future, monitors of health aspects belonging to home residents will sustain their healthy life daily. Keywords: smart home, multi-agent system, K-Nearest Neighbor algorithm, K-Means Clustering algorithm 1. Introduction Smart home, also known as an intelligent house, incorporates special devices that manage house features.

Abstract: We study the multi-agent Bayesian optimization (BO) problem, where multiple agents maximize a black-box function via iterative queries. We focus on Entropy Search (ES), a sample-efficient BO algorithm that selects queries to maximize the mutual information about the maximum of the black-box function. One of the main challenges of ES is that calculating the mutual information requires computationally-costly approximation techniques. For multi-agent BO problems, the computational cost of ES is exponential in the number of agents. To address this challenge, we propose the Gaussian Max-value Entropy Search, a multi-agent BO algorithm with favorable sample and computational efficiency.

In distributed model predictive control (MPC), the control input at each sampling time is computed by solving a large-scale optimal control problem (OCP) over a finite horizon using distributed algorithms. Typically, such algorithms require several (virtually, infinite) communication rounds between the subsystems to converge, which is a major drawback both computationally and from an energetic perspective (for wireless systems). Motivated by these challenges, we propose a suboptimal distributed MPC scheme in which the total communication burden is distributed also in time, by maintaining a running solution estimate for the large-scale OCP and updating it at each sampling time. We demonstrate that, under some regularity conditions, the resulting suboptimal MPC control law recovers the qualitative robust stability properties of optimal MPC, if the communication budget at each sampling time is large enough.

Recent papers have introduced a novel approach to explain why a Predictive Process Monitoring (PPM) model for outcome-oriented predictions provides wrong predictions. Moreover, they have shown how to exploit the explanations, obtained using state-of-the art post-hoc explainers, to identify the most common features that induce a predictor to make mistakes in a semi-automated way, and, in turn, to reduce the impact of those features and increase the accuracy of the predictive model. This work starts from the assumption that frequent control flow patterns in event logs may represent important features that characterize, and therefore explain, a certain prediction. Therefore, in this paper, we (i) employ a novel encoding able to leverage DECLARE constraints in Predictive Process Monitoring and compare the effectiveness of this encoding with Predictive Process Monitoring state-of-the art encodings, in particular for the task of outcome-oriented predictions; (ii) introduce a completely automated pipeline for the identification of the most common features inducing a predictor to make mistakes; and (iii) show the effectiveness of the proposed pipeline in increasing the accuracy of the predictive model by validating it on different real-life datasets.

Learning to predict agent motions with relationship reasoning is important for many applications. In motion prediction tasks, maintaining motion equivariance under Euclidean geometric transformations and invariance of agent interaction is a critical and fundamental principle. However, such equivariance and invariance properties are overlooked by most existing methods. To fill this gap, we propose EqMotion, an efficient equivariant motion prediction model with invariant interaction reasoning. To achieve motion equivariance, we propose an equivariant geometric feature learning module to learn a Euclidean transformable feature through dedicated designs of equivariant operations. To reason agent's interactions, we propose an invariant interaction reasoning module to achieve a more stable interaction modeling. To further promote more comprehensive motion features, we propose an invariant pattern feature learning module to learn an invariant pattern feature, which cooperates with the equivariant geometric feature to enhance network expressiveness. We conduct experiments for the proposed model on four distinct scenarios: particle dynamics, molecule dynamics, human skeleton motion prediction and pedestrian trajectory prediction. Experimental results show that our method is not only generally applicable, but also achieves state-of-the-art prediction performances on all the four tasks, improving by 24.0/30.1/8.6/9.2%. Code is available at https://github.com/MediaBrain-SJTU/EqMotion.

This paper introduces a new concept. We intend to give life to a software agent. A software agent is a computer program that acts on a user's behalf. We put a DNA inside the agent. DNA is a simple text, a whole roadmap of a network of agents or a system with details. A Dynamic Numerical Abstract of a multiagent system. It is also a reproductive part for an \emph{agent} that makes the agent take actions and decide independently and reproduce coworkers. By defining different DNA structures, one can establish new agents and different nets for different usages. We initiate such thinking as \emph{DNA programming}. This strategy leads to a new field of programming. This type of programming can help us manage large systems with various elements with an incredibly organized customizable structure. An agent can reproduce another agent. We put one or a few agents around a given network, and the agents will reproduce themselves till they can reach others and pervade the whole network. An agent's position or other environmental or geographical characteristics make it possible for an agent to know its active set of \emph{genes} on its DNA. The active set of genes specifies its duties. There is a database that includes a list of functions s.t. each one is an implementation of what a \emph{gene} represents. To utilize a decentralized database, we may use a blockchain-based structure. This design can adapt to a system that manages many static and dynamic networks. This network could be a distributed system, a decentralized system, a telecommunication network such as a 5G monitoring system, an IoT management system, or even an energy management system. The final system is the combination of all the agents and the overlay net that connects the agents. We denote the final net as the \emph{body} of the system.

Energetic materials (EM) cover a wide spectrum of propellants, pyrotechnics, and explosives and are key components in military applications for propulsion and munition systems and in civilian applications such as construction and mining [1]. Heterogenous/composite EMs have complex microstructures which significantly influence--along with chemistry--the property and performance of these materials [2-8]. There is increasing research interest in controlling the microstructure of EM, to engineer their properties and performance for targeted functional specificity [9-10]. EMs are typically solid-solid composites of organic energetic crystals (commonly CHNO compounds), inclusions (i.e., metals, nanoparticles), and plastic binders. The CHNO materials are commonly categorized based on how sensitive they are to an external load/mechanical insult. They can range f rom'insensitive' (such as TATB - based EMs [11]) to'highly sensitive' (PETN-based EMs [12-13]) with others such as HMX, CL-20, and RDX ranging in between [14]. The sensitivity is closely connected with the molecular structure of these species of EMs within the CHNO family. However, when they are formed into propellants and explosives, the sensitivity is also impacted by the physical structure, composition, and formulation of the material mixtures, as reviewed by Handley et al. [1]. In other words, the design of a mixture and its microstructure can define the overall properties and performance characteristics of formed EM, thus opening the possibility of systematic methods to engineer materials by their design.

Adversarial robustness assessment for video recognition models has raised concerns owing to their wide applications on safety-critical tasks. Compared with images, videos have much high dimension, which brings huge computational costs when generating adversarial videos. This is especially serious for the query-based black-box attacks where gradient estimation for the threat models is usually utilized, and high dimensions will lead to a large number of queries. To mitigate this issue, we propose to simultaneously eliminate the temporal and spatial redundancy within the video to achieve an effective and efficient gradient estimation on the reduced searching space, and thus query number could decrease. To implement this idea, we design the novel Adversarial spatial-temporal Focus (AstFocus) attack on videos, which performs attacks on the simultaneously focused key frames and key regions from the inter-frames and intra-frames in the video. AstFocus attack is based on the cooperative Multi-Agent Reinforcement Learning (MARL) framework. One agent is responsible for selecting key frames, and another agent is responsible for selecting key regions. These two agents are jointly trained by the common rewards received from the black-box threat models to perform a cooperative prediction. By continuously querying, the reduced searching space composed of key frames and key regions is becoming precise, and the whole query number becomes less than that on the original video. Extensive experiments on four mainstream video recognition models and three widely used action recognition datasets demonstrate that the proposed AstFocus attack outperforms the SOTA methods, which is prevenient in fooling rate, query number, time, and perturbation magnitude at the same.