In Towards a theory of superminds, I describe a theory of collective intelligence based on the active inference framework pioneered by Karl Friston. As required by active inference, that theory implies that all collectives (such as teams and organizations) operate on the basis of an implicit collective model of the world -- which provides them with the ability to make sense of observations and to predict outcomes of alternative courses of action ("policies" in Friston-speak). If we buy into the theorem that any Markov blanket (causally atomic subsystem) can always be described as performing some form of approximate active inference, this isn't polemical at all. Yet, as I've come to understand, there is something deeply counterintuitive in the conclusion. The fact that needs to be explained is that, unlike people, animals, unicellular beings, and even artificial agents, collectives are composed of clearly distinct and autonomous constituents -- and yet are capable of acting as collectives. This collective action can be as simple as the coherent motion of a flock of birds, or as complex as the financial markets incorporating sophisticated information about expectations of the future into asset prices (or for that matter, a corporation acting out a complex business strategy and production structure); the salient point is that the constituents are autonomous (and, at least for the examples involving humans, will tell you they follow their own free will), and yet the collective can be very accurately described as an agent in its own right. It turns out that fully resolving this apparent contradiction requires gaining a thorough understanding of what these models are, which I will attempt to do in this post.

Reinforcement learning is a promising approach to learning control policies for performing complex multi-agent robotics tasks. However, a policy learned in simulation often fails to guarantee even simple safety properties such as obstacle avoidance. To ensure safety, we propose multi-agent model predictive shielding (MAMPS), an algorithm that provably guarantees safety for an arbitrary learned policy. In particular, it operates by using the learned policy as often as possible, but instead uses a backup policy in cases where it cannot guarantee the safety of the learned policy. Using a multi-agent simulation environment, we show how MAMPS can achieve good performance while ensuring safety.

Nature provides us with abundant examples of how large numbers of individuals can make decisions without the coordination of a central authority. Social insects, birds, fishes, and many other living collectives, rely on simple interaction mechanisms to do so. They individually gather information from the environment; small bits of a much larger picture that are then shared locally among the members of the collective and processed together to output a commonly agreed choice. Throughout evolution, Nature found solutions to collective decision-making problems that are intriguing to engineers for their robustness to malfunctioning or lost individuals, their flexibility in face of dynamic environments, and their ability to scale with large numbers of members. In the last decades, whereas biologists amassed large amounts of experimental evidence, engineers took inspiration from these and other examples to design distributed algorithms that, while maintaining the same properties of their natural counterparts, come with guarantees on their performance in the form of predictive mathematical models. In this paper, we review the fundamental processes that lead to a collective decision. We discuss examples of collective decisions in biological systems and show how similar processes can be engineered to design artificial ones. During this journey, we review a framework to design distributed decision-making algorithms that are modular, can be instantiated and extended in different ways, and are supported by a suit of predictive mathematical models.

Machine learning algorithms--also known as artificial intelligence (AI)--can better detect sleep apnea compared with traditional linear approaches, according to a study being presented at the CHEST Annual Meeting 2019. The researchers included 620 patients who were referred to a sleep lab in a suburban community sleep center. Researchers collected information on 12 select parameters: height, weight, waist, hip, body mass index, age, neck side, Modified Friedman stage, snoring, Epworth sleepiness scale, sex, and daytime sleepiness. During phase I, researchers used a binary particle swarm optimization technique to select the best sub-features that characterize sleep apnea. In phase II, they built an artificial neural network model based on a feedforward algorithm to detect sleep apnea.

In typical reinforcement learning (RL), the environment is assumed given and the goal of the learning is to identify an optimal policy for the agent taking actions through its interactions with the environment. In this paper, we extend this setting by considering the environment is not given, but controllable and learnable through its interaction with the agent at the same time. This extension is motivated by environment design scenarios in the real-world, including game design, shopping space design and traffic signal design. Theoretically, we find a dual Markov decision process (MDP) w.r.t. the environment to that w.r.t. the agent, and derive a policy gradient solution to optimizing the parametrized environment. Furthermore, discontinuous environments are addressed by a proposed general generative framework. Our experiments on a Maze game design task show the effectiveness of the proposed algorithms in generating diverse and challenging Mazes against various agent settings.

A shield is attached to a system to guarantee safety by correcting the system's behavior at runtime. Existing methods that employ design-time synthesis of shields do not scale to multi-agent systems. Moreover, such shields are typically implemented in a centralized manner, requiring global information on the state of all agents in the system. We address these limitations through a new approach where the shields are synthesized at runtime and do not require global information. There is a shield onboard every agent, which can only modify the behavior of the corresponding agent. In this approach, which is fundamentally decentralized, the shield on every agent has two components: a pathfinder that corrects the behavior of the agent and an ordering mechanism that dynamically modifies the priority of the agent. The current priority determines if the shield uses the pathfinder to modify behavior of the agent. We derive an upper bound on the maximum deviation for any agent from its original behavior. We prove that the worst-case synthesis time is quadratic in the number of agents at runtime as opposed to exponential at design-time for existing methods. We test the performance of the decentralized, runtime shield synthesis approach on a collision-avoidance problem. For 50 agents in a 50x50 grid, the synthesis at runtime requires a few seconds per agent whenever a potential collision is detected. In contrast, the centralized design-time synthesis of shields for a similar setting is intractable beyond 4 agents in a 5x5 grid.

We present a novel algorithm (DeepMNavigate) for global multi-agent navigation in dense scenarios using deep reinforcement learning. Our approach uses local and global information for each robot based on motion information maps. We use a three-layer CNN that uses these maps as input and generate a suitable action to drive each robot to its goal position. Our approach is general, learns an optimal policy using a multi-scenario, multi-state training algorithm, and can directly handle raw sensor measurements for local observations. We demonstrate the performance on complex, dense benchmarks with narrow passages on environments with tens of agents. We highlight the algorithm's benefits over prior learning methods and geometric decentralized algorithms in complex scenarios.

In the current level of evolution of Soccer 3D, motion control is a key factor in team's performance. Recent works takes advantages of model-free approaches based on Machine Learning to exploit robot dynamics in order to obtain faster locomotion skills, achieving running policies and, therefore, opening a new research direction in the Soccer 3D environment. In this work, we present a methodology based on Deep Reinforcement Learning that learns running skills without any prior knowledge, using a neural network whose inputs are related to robot's dynamics. Our results outperformed the previous state-of-the-art sprint velocity reported in Soccer 3D literature by a significant margin. It also demonstrated improvement in sample efficiency, being able to learn how to run in just few hours. We reported our results analyzing the training procedure and also evaluating the policies in terms of speed, reliability and human similarity. Finally, we presented key factors that lead us to improve previous results and shared some ideas for future work.

Despite of the recent progress in agents that learn through interaction, there are several challenges in terms of sample efficiency and generalization across unseen behaviors during training. To mitigate these problems, we propose and apply a first-order Meta-Learning algorithm called Bottom-Up Meta-Policy Search (BUMPS), which works with two-phase optimization procedure: firstly, in a meta-training phase, it distills few expert policies to create a meta-policy capable of generalizing knowledge to unseen tasks during training; secondly, it applies a fast adaptation strategy named Policy Filtering, which evaluates few policies sampled from the meta-policy distribution and selects which best solves the task. We conducted all experiments in the RoboCup 3D Soccer Simulation domain, in the context of kick motion learning. We show that, given our experimental setup, BUMPS works in scenarios where simple multi-task Reinforcement Learning does not. Finally, we performed experiments in a way to evaluate each component of the algorithm.

Systems engineering processes coordinate the effort of different individuals to generate a product satisfying certain requirements. As the involved engineers are self-interested agents, the goals at different levels of the systems engineering hierarchy may deviate from the system-level goals which may cause budget and schedule overruns. Therefore, there is a need of a systems engineering theory that accounts for the human behavior in systems design. To this end, the objective of this paper is to develop and analyze a principal-agent model of a one-shot (single iteration), shallow (one level of hierarchy) systems engineering process. We assume that the systems engineer maximizes the expected utility of the system, while the subsystem engineers seek to maximize their expected utilities. Furthermore, the systems engineer is unable to monitor the effort of the subsystem engineer and may not have a complete information about their types or the complexity of the design task. However, the systems engineer can incentivize the subsystem engineers by proposing specific contracts. To obtain an optimal incentive, we pose and solve numerically a bi-level optimization problem. Through extensive simulations, we study the optimal incentives arising from different system-level value functions under various combinations of effort costs, problem-solving skills, and task complexities.