Our research aims to develop intelligent collaborative agents that are human-aware - they can model, learn, and reason about their human partner's physiological, cognitive, and affective states. In this paper, we study how adaptive coaching interactions can be designed to help people develop sustainable healthy behaviors. We leverage the common model of cognition - CMC [26] - as a framework for unifying several behavior change theories that are known to be useful in human-human coaching. We motivate a set of interactive system desiderata based on the CMC-based view of behavior change. Then, we propose PARCoach - an interactive system that addresses the desiderata. PARCoach helps a trainee pick a relevant health goal, set an implementation intention, and track their behavior. During this process, the trainee identifies a specific goal-directed behavior as well as the situational context in which they will perform it. PARCcoach uses this information to send notifications to the trainee, reminding them of their chosen behavior and the context. We report the results from a 4-week deployment with 60 participants. Our results support the CMC-based view of behavior change and demonstrate that the desiderata for proposed interactive system design is useful in producing behavior change.

In Towards a theory of superminds, I describe a theory of collective intelligence based on the active inference framework pioneered by Karl Friston. As required by active inference, that theory implies that all collectives (such as teams and organizations) operate on the basis of an implicit collective model of the world -- which provides them with the ability to make sense of observations and to predict outcomes of alternative courses of action ("policies" in Friston-speak). If we buy into the theorem that any Markov blanket (causally atomic subsystem) can always be described as performing some form of approximate active inference, this isn't polemical at all. Yet, as I've come to understand, there is something deeply counterintuitive in the conclusion. The fact that needs to be explained is that, unlike people, animals, unicellular beings, and even artificial agents, collectives are composed of clearly distinct and autonomous constituents -- and yet are capable of acting as collectives. This collective action can be as simple as the coherent motion of a flock of birds, or as complex as the financial markets incorporating sophisticated information about expectations of the future into asset prices (or for that matter, a corporation acting out a complex business strategy and production structure); the salient point is that the constituents are autonomous (and, at least for the examples involving humans, will tell you they follow their own free will), and yet the collective can be very accurately described as an agent in its own right. It turns out that fully resolving this apparent contradiction requires gaining a thorough understanding of what these models are, which I will attempt to do in this post.

In Towards a theory of superminds, I describe a theory of collective intelligence based on the active inference framework pioneered by Karl Friston. As required by active inference, that theory implies that all collectives (such as teams and organizations) operate on the basis of an implicit collective model of the world -- which provides them with the ability to make sense of observations and to predict outcomes of alternative courses of action ("policies" in Friston-speak). If we buy into the theorem that any Markov blanket (causally atomic subsystem) can always be described as performing some form of approximate active inference, this isn't polemical at all. Yet, as I've come to understand, there is something deeply counterintuitive in the conclusion. The fact that needs to be explained is that, unlike people, animals, unicellular beings, and even artificial agents, collectives are composed of clearly distinct and autonomous constituents -- and yet are capable of acting as collectives. This collective action can be as simple as the coherent motion of a flock of birds, or as complex as the financial markets incorporating sophisticated information about expectations of the future into asset prices (or for that matter, a corporation acting out a complex business strategy and production structure); the salient point is that the constituents are autonomous (and, at least for the examples involving humans, will tell you they follow their own free will), and yet the collective can be very accurately described as an agent in its own right. It turns out that fully resolving this apparent contradiction requires gaining a thorough understanding of what these models are, which I will attempt to do in this post.

Reinforcement learning is a promising approach to learning control policies for performing complex multi-agent robotics tasks. However, a policy learned in simulation often fails to guarantee even simple safety properties such as obstacle avoidance. To ensure safety, we propose multi-agent model predictive shielding (MAMPS), an algorithm that provably guarantees safety for an arbitrary learned policy. In particular, it operates by using the learned policy as often as possible, but instead uses a backup policy in cases where it cannot guarantee the safety of the learned policy. Using a multi-agent simulation environment, we show how MAMPS can achieve good performance while ensuring safety.

Nature provides us with abundant examples of how large numbers of individuals can make decisions without the coordination of a central authority. Social insects, birds, fishes, and many other living collectives, rely on simple interaction mechanisms to do so. They individually gather information from the environment; small bits of a much larger picture that are then shared locally among the members of the collective and processed together to output a commonly agreed choice. Throughout evolution, Nature found solutions to collective decision-making problems that are intriguing to engineers for their robustness to malfunctioning or lost individuals, their flexibility in face of dynamic environments, and their ability to scale with large numbers of members. In the last decades, whereas biologists amassed large amounts of experimental evidence, engineers took inspiration from these and other examples to design distributed algorithms that, while maintaining the same properties of their natural counterparts, come with guarantees on their performance in the form of predictive mathematical models. In this paper, we review the fundamental processes that lead to a collective decision. We discuss examples of collective decisions in biological systems and show how similar processes can be engineered to design artificial ones. During this journey, we review a framework to design distributed decision-making algorithms that are modular, can be instantiated and extended in different ways, and are supported by a suit of predictive mathematical models.

Machine learning algorithms--also known as artificial intelligence (AI)--can better detect sleep apnea compared with traditional linear approaches, according to a study being presented at the CHEST Annual Meeting 2019. The researchers included 620 patients who were referred to a sleep lab in a suburban community sleep center. Researchers collected information on 12 select parameters: height, weight, waist, hip, body mass index, age, neck side, Modified Friedman stage, snoring, Epworth sleepiness scale, sex, and daytime sleepiness. During phase I, researchers used a binary particle swarm optimization technique to select the best sub-features that characterize sleep apnea. In phase II, they built an artificial neural network model based on a feedforward algorithm to detect sleep apnea.

In typical reinforcement learning (RL), the environment is assumed given and the goal of the learning is to identify an optimal policy for the agent taking actions through its interactions with the environment. In this paper, we extend this setting by considering the environment is not given, but controllable and learnable through its interaction with the agent at the same time. This extension is motivated by environment design scenarios in the real-world, including game design, shopping space design and traffic signal design. Theoretically, we find a dual Markov decision process (MDP) w.r.t. the environment to that w.r.t. the agent, and derive a policy gradient solution to optimizing the parametrized environment. Furthermore, discontinuous environments are addressed by a proposed general generative framework. Our experiments on a Maze game design task show the effectiveness of the proposed algorithms in generating diverse and challenging Mazes against various agent settings.

A shield is attached to a system to guarantee safety by correcting the system's behavior at runtime. Existing methods that employ design-time synthesis of shields do not scale to multi-agent systems. Moreover, such shields are typically implemented in a centralized manner, requiring global information on the state of all agents in the system. We address these limitations through a new approach where the shields are synthesized at runtime and do not require global information. There is a shield onboard every agent, which can only modify the behavior of the corresponding agent. In this approach, which is fundamentally decentralized, the shield on every agent has two components: a pathfinder that corrects the behavior of the agent and an ordering mechanism that dynamically modifies the priority of the agent. The current priority determines if the shield uses the pathfinder to modify behavior of the agent. We derive an upper bound on the maximum deviation for any agent from its original behavior. We prove that the worst-case synthesis time is quadratic in the number of agents at runtime as opposed to exponential at design-time for existing methods. We test the performance of the decentralized, runtime shield synthesis approach on a collision-avoidance problem. For 50 agents in a 50x50 grid, the synthesis at runtime requires a few seconds per agent whenever a potential collision is detected. In contrast, the centralized design-time synthesis of shields for a similar setting is intractable beyond 4 agents in a 5x5 grid.

We present a novel algorithm (DeepMNavigate) for global multi-agent navigation in dense scenarios using deep reinforcement learning. Our approach uses local and global information for each robot based on motion information maps. We use a three-layer CNN that uses these maps as input and generate a suitable action to drive each robot to its goal position. Our approach is general, learns an optimal policy using a multi-scenario, multi-state training algorithm, and can directly handle raw sensor measurements for local observations. We demonstrate the performance on complex, dense benchmarks with narrow passages on environments with tens of agents. We highlight the algorithm's benefits over prior learning methods and geometric decentralized algorithms in complex scenarios.

In the current level of evolution of Soccer 3D, motion control is a key factor in team's performance. Recent works takes advantages of model-free approaches based on Machine Learning to exploit robot dynamics in order to obtain faster locomotion skills, achieving running policies and, therefore, opening a new research direction in the Soccer 3D environment. In this work, we present a methodology based on Deep Reinforcement Learning that learns running skills without any prior knowledge, using a neural network whose inputs are related to robot's dynamics. Our results outperformed the previous state-of-the-art sprint velocity reported in Soccer 3D literature by a significant margin. It also demonstrated improvement in sample efficiency, being able to learn how to run in just few hours. We reported our results analyzing the training procedure and also evaluating the policies in terms of speed, reliability and human similarity. Finally, we presented key factors that lead us to improve previous results and shared some ideas for future work.