When developing autonomous systems, engineers and other stakeholders make great effort to prepare the system for all foreseeable events and conditions. However, these systems are still bound to encounter events and conditions that were not considered at design time. For reasons like safety, cost, or ethics, it is often highly desired that these new situations be handled correctly upon first encounter. In this paper we first justify our position that there will always exist unpredicted events and conditions, driven among others by: new inventions in the real world; the diversity of world-wide system deployments and uses; and, the non-negligible probability that multiple seemingly unlikely events, which may be neglected at design time, will not only occur, but occur together. We then argue that despite this unpredictability property, handling these events and conditions is indeed possible. Hence, we offer and exemplify design principles that when applied in advance, can enable systems to deal, in the future, with unpredicted circumstances. We conclude with a discussion of how this work and a broader theoretical study of the unexpected can contribute toward a foundation of engineering principles for developing trustworthy next-generation autonomous systems.

Technical singularity is defined as a hypothetical future of superhuman machines with a cognitive capability far beyond the capacity of human minds. In the journey toward this potential technology revolution is something that I have been focused on called artificial swarm intelligence. A starling murmuration, something that people have told me is awe-inspiring, is a marvel of nature similar to an army of ants or a swarm of bees. How do all these individual entities organize around a common mission that includes a form of collaboration and unified orchestration as a team? When thinking about swarms of AI bots or even nanobots, the foundational concept we want to define is what exactly AI bot are.

MagNet: Discovering Multi-agent Interaction Dynamics using Neural Network Priyabrata Saha, Arslan Ali, Burhan A. Mudassar, Y un Long and Saibal Mukhopadhyay Abstract -- We present the MagNet, a multi-agent interaction network to discover governing dynamics and predict evolution of a complex system from observations. We formulate a multi-agent system as a coupled nonlinear network with a generic ordinary differential equation (ODE) based state evolution, and develop a neural network based realization of its time-discretized model. MagNet is trained to discover the core dynamics of a multi-agent system from observations, and tuned online to learn agent-specific parameters of the dynamics to ensure accurate prediction even when physical or relational attributes of agents, or number of agents change. We evaluate MagNet on point-mass system in two-dimensional space, Ku-ramoto phase synchronization dynamics and predator-swarm interaction dynamics demonstrating orders of magnitude improvement in prediction accuracy over traditional deep learning models. I NTRODUCTION Multi-agent systems are prevalent in both the natural world and engineered world. Engineered distributed systems of mobile robots, multiple sensors, unmanned aerial vehicles etc. often take inspiration from natural multi-agent systems like swarms, schools, flocks, and herds of social animals or birds. Understanding the behavior of such natural or engineered multi-agent systems from sensory observations is a key challenge in robotics from the design and adversarial perspective. Discovering the hidden dynamics of a multi-agent interaction from observations will enable machines to simulate and predict evolution of complex systems.

The capability and spread of such systems have reached the point where they are beginning to touch much of everyday life. However, regulators grapple with how to deal with autonomous systems, for example how could we certify an Unmanned Aerial System for autonomous use in civilian airspace? We here analyse what is needed in order to provide verified reliable behaviour of an autonomous system, analyse what can be done as the state-of-the-art in automated verification, and propose a roadmap towards developing regulatory guidelines, including articulating challenges to researchers, to engineers, and to regulators. Case studies in seven distinct domains illustrate the article. Keywords: autonomous systems; certification; verification; Artificial Intelligence 1 Introduction Since the dawn of human history, humans have designed, implemented and adopted tools to make it easier to perform tasks, often improving efficiency, safety, or security.

Recent findings in neuroscience suggest that the human brain represents information in a geometric structure (for instance, through conceptual spaces). In order to communicate, we flatten the complex representation of entities and their attributes into a single word or a sentence. In this paper we use graph convolutional networks to support the evolution of language and cooperation in multi-agent systems. Motivated by an image-based referential game, we propose a graph referential game with varying degrees of complexity, and we provide strong baseline models that exhibit desirable properties in terms of language emergence and cooperation. We show that the emerged communication protocol is robust, that the agents uncover the true factors of variation in the game, and that they learn to generalize beyond the samples encountered during training.

We live in an era when humans are busy training a new intelligence on this planet. Every once in a while, researchers come up with a novel way to speed up that teaching process. That's what happened at Microsoft Research where computer scientists recently developed a new approach to using human emotion to train machines how to learn.[i] The research used virtual agents to facilitate learning various tasks in a simulated environment. What is most significant about this research is that it trained those agents by exposing them to the smiles of human subjects as they interacted with the system.

In this study into the player's emotional theory of mind of gameplaying agents, we investigate how an agent's behaviour and the player's own performance and emotions shape the recognition of a frustrated behaviour. We focus on the perception of frustration as it is a prevalent affective experience in human-computer interaction. We present a testbed game tailored towards this end, in which a player competes against an agent with a frustration model based on theory. We collect gameplay data, an annotated ground truth about the player's appraisal of the agent's frustration, and apply face recognition to estimate the player's emotional state. We examine the collected data through correlation analysis and predictive machine learning models, and find that the player's observable emotions are not correlated highly with the perceived frustration of the agent. This suggests that our subject's theory of mind is a cognitive process based on the gameplay context. Our predictive models---using ranking support vector machines---corroborate these results, yielding moderately accurate predictors of players' theory of mind.

Usually two or more agents play a cooperative game where agents' goals are common but they access different information. In order to solve the given task agents have to share useful information with each other through a discrete bottleneck called the communication channel. The discrete symbols in the message do not have any a priori meaning but agents learn to cooperate by attributing meaning to the messages; a language protocol emerges as a byproduct of the training process. This emergent language serves only one goal: to complete the task successfully. One of the promises of this approach is to provide meaningful insights into the early stages of human language emergence as a result of cooperation.

Cooperative Multi-agent Reinforcement Learning (MARL) is crucial for cooperative decentralized decision learning in many domains such as search and rescue, drone surveillance, package delivery and fire fighting problems. In these domains, a key challenge is learning with a few good experiences, i.e., positive reinforcements are obtained only in a few situations (e.g., on extinguishing a fire or tracking a crime or delivering a package) and in most other situations there is zero or negative reinforcement. Learning decisions with a few good experiences is extremely challenging in cooperative MARL problems due to three reasons. First, compared to the single agent case, exploration is harder as multiple agents have to be coordinated to receive a good experience. Second, environment is not stationary as all the agents are learning at the same time (and hence change policies). Third, scale of problem increases significantly with every additional agent. Relevant existing work is extensive and has focussed on dealing with a few good experiences in single-agent RL problems or on scalable approaches for handling non-stationarity in MARL problems. Unfortunately, neither of these approaches (or their extensions) are able to address the problem of sparse good experiences effectively. Therefore, we provide a novel fictitious self imitation approach that is able to simultaneously handle non-stationarity and sparse good experiences in a scalable manner. Finally, we provide a thorough comparison (experimental or descriptive) against relevant cooperative MARL algorithms to demonstrate the utility of our approach.

A negotiation process by 2 agents e1 and e2 can be interleaved by another negotiation process between, say, e1 and e3. The interleaving may alter the resource allocation assumed at the inception of the first negotiation process. Existing proposals for argumentation-based negotiations have focused primarily on two-agent bilateral negotiations, but scarcely on the concurrency of multi-agent negotiations. To fill the gap, we present a novel argumentation theory, basing its development on abstract persuasion argumentation (which is an abstract argumentation formalism with a dynamic relation). Incorporating into it numerical information and a mechanism of handshakes among members of the dynamic relation, we show that the extended theory adapts well to concurrent multi-agent negotiations over scarce resources.