As part of Microsoft's research into ways to use machine learning and AI to improve security defenses, the company has released an open source attack toolkit to let researchers create simulated network environments and see how they fare against attacks. Microsoft 365 Defender Research released CyberBattleSim, which creates a network simulation and models how threat actors can move laterally through the network looking for weak points. When building the attack simulation, enterprise defenders and researchers create various nodes on the network and indicate which services are running, which vulnerabilities are present, and what type of security controls are in place. Automated agents, representing threat actors, are deployed in the attack simulation to randomly execute actions as they try to take over the nodes. "The simulated attacker's goal is to take ownership of some portion of the network by exploiting these planted vulnerabilities. While the simulated attacker moves through the network, a defender agent watches the network activity to detect the presence of the attacker and contain the attack," the Microsoft 365 Defender Research Team wrote in a post discussing the project.

With the increasing scale, complexity, and heterogeneity of the next generation networked systems, seamless control, management, and security of such systems becomes increasingly challenging. Many diverse applications have driven interest in networked systems, including large-scale distributed learning, multi-agent optimization, 5G service provisioning, and network slicing, etc. In this paper, we propose trust as a measure to evaluate the status of network agents and improve the decision-making process. We interpret trust as a relation among entities that participate in various protocols. Trust relations are based on evidence created by the interactions of entities within a protocol and may be a composite of multiple metrics such as availability, reliability, resilience, etc. depending on application context. We first elaborate on the importance of trust as a metric and then present a mathematical framework for trust computation and aggregation within a network. Then we show in practice, how trust can be integrated into network decision-making processes by presenting two examples. In the first example, we show how utilizing the trust evidence can improve the performance and the security of Federated Learning. Second, we show how a 5G network resource provisioning framework can be improved when augmented with a trust-aware decision-making scheme. We verify the validity of our trust-based approach through simulations. Finally, we explain the challenges associated with aggregating the trust evidence and briefly explain our ideas to tackle them.

Joint attention - the ability to purposefully coordinate attention with another agent, and mutually attend to the same thing -- is a critical component of human social cognition. In this paper, we ask whether joint attention can be useful as a mechanism for improving multi-agent coordination and social learning. We first develop deep reinforcement learning (RL) agents with a recurrent visual attention architecture. We then train agents to minimize the difference between the attention weights that they apply to the environment at each timestep, and the attention of other agents. Our results show that this joint attention incentive improves agents' ability to solve difficult coordination tasks, by reducing the exponential cost of exploring the joint multi-agent action space. Joint attention leads to higher performance than a competitive centralized critic baseline across multiple environments. Further, we show that joint attention enhances agents' ability to learn from experts present in their environment, even when completing hard exploration tasks that do not require coordination. Taken together, these findings suggest that joint attention may be a useful inductive bias for multi-agent learning.

Natural languages commonly display a trade-off among different strategies to convey constituent roles. A similar trade-off, however, has not been observed in recent simulations of iterated language learning with neural network based agents (Chaabouni et al., 2019b). In this work, we re-evaluate this result in the light of two important factors, namely: the lack of effort-based pressure in the agents and the lack of variability in the initial input language.

When intelligent agents communicate to accomplish shared goals, how do these goals shape the agents' language? We study the dynamics of learning in latent language policies (LLPs), in which instructor agents generate natural-language subgoal descriptions and executor agents map these descriptions to low-level actions. LLPs can solve challenging long-horizon reinforcement learning problems and provide a rich model for studying task-oriented language use. But previous work has found that LLP training is prone to semantic drift (use of messages in ways inconsistent with their original natural language meanings). Here, we demonstrate theoretically and empirically that multitask training is an effective counter to this problem: we prove that multitask training eliminates semantic drift in a well-studied family of signaling games, and show that multitask training of neural LLPs in a complex strategy game reduces drift and while improving sample efficiency.

While deep reinforcement learning (RL) promises freedom from hand-labeled data, great successes, especially for Embodied AI, require significant work to create supervision via carefully shaped rewards. Indeed, without shaped rewards, i.e., with only terminal rewards, present-day Embodied AI results degrade significantly across Embodied AI problems from single-agent Habitat-based PointGoal Navigation (SPL drops from 55 to 0) and two-agent AI2-THOR-based Furniture Moving (success drops from 58% to 1%) to three-agent Google Football-based 3 vs. 1 with Keeper (game score drops from 0.6 to 0.1). As training from shaped rewards doesn't scale to more realistic tasks, the community needs to improve the success of training with terminal rewards. For this we propose GridToPix: 1) train agents with terminal rewards in gridworlds that generically mirror Embodied AI environments, i.e., they are independent of the task; 2) distill the learned policy into agents that reside in complex visual worlds. Despite learning from only terminal rewards with identical models and RL algorithms, GridToPix significantly improves results across tasks: from PointGoal Navigation (SPL improves from 0 to 64) and Furniture Moving (success improves from 1% to 25%) to football gameplay (game score improves from 0.1 to 0.6). GridToPix even helps to improve the results of shaped reward training.

This paper develops a natural-language agent-based model of argumentation (ABMA). Its artificial deliberative agents (ADAs) are constructed with the help of so-called neural language models recently developed in AI and computational linguistics. ADAs are equipped with a minimalist belief system and may generate and submit novel contributions to a conversation. The natural-language ABMA allows us to simulate collective deliberation in English, i.e. with arguments, reasons, and claims themselves -- rather than with their mathematical representations (as in formal models). This paper uses the natural-language ABMA to test the robustness of formal reason-balancing models of argumentation [Maes & Flache 2013, Singer et al. 2019]: First of all, as long as ADAs remain passive, confirmation bias and homophily updating trigger polarization, which is consistent with results from formal models. However, once ADAs start to actively generate new contributions, the evolution of a conservation is dominated by properties of the agents *as authors*. This suggests that the creation of new arguments, reasons, and claims critically affects a conversation and is of pivotal importance for understanding the dynamics of collective deliberation. The paper closes by pointing out further fruitful applications of the model and challenges for future research.

This paper presents a new algorithm named spherical vector-based particle swarm optimization (SPSO) to deal with the problem of path planning for unmanned aerial vehicles (UAVs) in complicated environments subjected to multiple threats. A cost function is first formulated to convert the path planning into an optimization problem that incorporates requirements and constraints for the feasible and safe operation of the UAV. SPSO is then used to find the optimal path that minimizes the cost function by efficiently searching the configuration space of the UAV via the correspondence between the particle position and the speed, turn angle and climb/dive angle of the UAV. To evaluate the performance of SPSO, eight benchmarking scenarios have been generated from real digital elevation model maps. The results show that the proposed SPSO outperforms not only other particle swarm optimization (PSO) variants including the classic PSO, phase angle-encoded PSO and quantum-behave PSO but also other state-of-the-art metaheuristic optimization algorithms including the genetic algorithm (GA), artificial bee colony (ABC), and differential evolution (DE) in most scenarios. In addition, experiments have been conducted to demonstrate the validity of the generated paths for real UAV operations. Source code of the algorithm can be found at https://github.com/duongpm/SPSO.

A key step in influence maximization in online social networks is the identification of a small number of users, known as influencers, who are able to spread influence quickly and widely to other users. The evolving nature of the topological structure of these networks makes it difficult to locate and identify these influencers. In this paper, we propose an adaptive agent-based evolutionary approach to address this problem in the context of both static and dynamic networks. This approach is shown to be able to adapt the solution as the network evolves. It is also applicable to large-scale networks due to its distributed framework. Evaluation of our approach is performed by using both synthetic networks and real-world datasets. Experimental results demonstrate that the proposed approach outperforms state-of-the-art seeding algorithms in terms of maximizing influence.

Since the number of elderly and patients who are in hospitals and healthcare centers are growing, providing efficient remote healthcare services seems very important. Currently, most such systems benefit from the distribution and autonomy features of multiagent systems and the structure of wireless sensor networks. On the one hand, securing the data of remote healthcare systems is one of the most significant concerns; particularly recent types of research about the security of remote healthcare systems keep them secure from eavesdropping and data modification. On the other hand, existing remote healthcare systems are still vulnerable against other common attacks of healthcare networks such as Denial of Service (DoS) and User to Root (U2R) attacks, because they are managed remotely and based on the Internet. Therefore, in this paper, we propose a secure framework for remote healthcare systems that consists of two phases. First, we design a healthcare system base on multiagent technology to collect data from a sensor network. Then, in the second phase, a layered architecture of intrusion detection systems that uses Support Vector Machine to learn the behavior of network traffic is applied. Based on our framework, we implement a secure remote healthcare system and evaluate this system against the frequent attacks of healthcare networks such as Smurf, Buffer overflow, Neptune, and Pod attacks. In the end, evaluation parameters of the layered architecture of intrusion detection systems prove the efficiency and correctness of our proposed framework.