Recently, autonomous driving has made substantial progress in addressing the most common traffic scenarios like intersection navigation and lane changing. However, most of these successes have been limited to scenarios with well-defined traffic rules and require minimal negotiation with other vehicles. In this paper, we introduce a previously unconsidered, yet everyday, high-conflict driving scenario requiring negotiations between agents of equal rights and priorities. There exists no centralized control structure and we do not allow communications. Therefore, it is unknown if other drivers are willing to cooperate, and if so to what extent. We train policies to robustly negotiate with opposing vehicles of an unobservable degree of cooperativeness using multi-agent reinforcement learning (MARL). We propose Discrete Asymmetric Soft Actor-Critic (DASAC), a maximum-entropy off-policy MARL algorithm allowing for centralized training with decentralized execution. We show that using DASAC we are able to successfully negotiate and traverse the scenario considered over 99% of the time. Our agents are robust to an unknown timing of opponent decisions, an unobservable degree of cooperativeness of the opposing vehicle, and previously unencountered policies. Furthermore, they learn to exhibit human-like behaviors such as defensive driving, anticipating solution options and interpreting the behavior of other agents.

This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge. Specifically, it presents the techniques utilized within the Tunnel (2019) and Urban (2020) competitions, where CoSTAR achieved 2nd and 1st place, respectively. We also discuss CoSTAR's demonstrations in Martian-analog surface and subsurface (lava tubes) exploration. The paper introduces our autonomy solution, referred to as NeBula (Networked Belief-aware Perceptual Autonomy). NeBula is an uncertainty-aware framework that aims at enabling resilient and modular autonomy solutions by performing reasoning and decision making in the belief space (space of probability distributions over the robot and world states). We discuss various components of the NeBula framework, including: (i) geometric and semantic environment mapping; (ii) a multi-modal positioning system; (iii) traversability analysis and local planning; (iv) global motion planning and exploration behavior; (i) risk-aware mission planning; (vi) networking and decentralized reasoning; and (vii) learning-enabled adaptation. We discuss the performance of NeBula on several robot types (e.g. wheeled, legged, flying), in various environments. We discuss the specific results and lessons learned from fielding this solution in the challenging courses of the DARPA Subterranean Challenge competition.

Academic research in board game playing AI has of course moved While artificial intelligence has been applied to control players' beyond most pedestrian board games, applying a diverse set of decisions in board games for over half a century, little attention algorithms for playing card games with millions of card combinations is given to games with no player competition. Pandemic is an exemplar such as Magic: the Gathering (Wizards of the Coast, 1993) [3], collaborative board game where all players coordinate to games of tactical card placement such as Lords of War (Black Box, overcome challenges posed by events occurring during the game's 2012) [19] and Carcassonne (Hans im Glück, 2000) [9], card games progression. This paper proposes an artificial agent which controls of team-based competition such as Hanabi (Abacusspiele, 2010) [26] all players' actions and balances chances of winning versus risk or Codenames (Czech Games Edition, 2015) [22], and many more. of losing in this highly stochastic environment. The agent applies Traditional board games such as chess [15] and backgammon a Rolling Horizon Evolutionary Algorithm on an abstraction of [23], as well as recent card games such as Race for the Galaxy (Rio the game-state that lowers the branching factor and simulates the Grande, 2007) [6] or digitized board games such as Hearthstone game's stochasticity. Results show that the proposed algorithm (Blizzard, 2014) [11, 18], focus on players competing to deplete another can find winning strategies more consistently in different games player's resources (pawns, hit points) or to accumulate more of varying difficulty.

Editor's note: This research was conducted by Sai Vemprala, Senior Researcher, and Ashish Kapoor, Partner Researcher, of Microsoft Research along with Sami Mian, who was a PhD Researcher at the University of Pittsburgh and an intern at Microsoft at the time of the work. Autonomous systems are composed of complex perception-action loops, where observations of the world need to be processed in real time to result in safe and effective actions. A significant amount of research has focused on creating perception and navigation algorithms for such systems, often using visual data from cameras to reason about which action to take depending on the platform and task at hand. While there have been a lot of improvements in how this reasoning is performed, and how information can be extracted efficiently from camera imagery, there are a number of challenges when it comes to achieving autonomous systems that receive and process information both accurately and quickly enough for applications in real-world scenarios. These challenges include the speed limitations posed by commercial off-the-shelf cameras, data that is unseen during training of vision models, and the limitations of sensors in RGB camera sensors.

The ability to perceive and reason about social interactions in the context of physical environments is core to human social intelligence and human-machine cooperation. However, no prior dataset or benchmark has systematically evaluated physically grounded perception of complex social interactions that go beyond short actions, such as high-fiving, or simple group activities, such as gathering. In this work, we create a dataset of physically-grounded abstract social events, PHASE, that resemble a wide range of real-life social interactions by including social concepts such as helping another agent. PHASE consists of 2D animations of pairs of agents moving in a continuous space generated procedurally using a physics engine and a hierarchical planner. Agents have a limited field of view, and can interact with multiple objects, in an environment that has multiple landmarks and obstacles. Using PHASE, we design a social recognition task and a social prediction task. PHASE is validated with human experiments demonstrating that humans perceive rich interactions in the social events, and that the simulated agents behave similarly to humans. As a baseline model, we introduce a Bayesian inverse planning approach, SIMPLE (SIMulation, Planning and Local Estimation), which outperforms state-of-the-art feed-forward neural networks. We hope that PHASE can serve as a difficult new challenge for developing new models that can recognize complex social interactions.

This paper presents a novel quasi-centralized approach for collision-free path planning of multi-robot systems (MRS) in obstacle-ridden environments. A new formation potential fields (FPF) concept is proposed around a virtual agent, located at the center of the formation which ensures self-organization and maintenance of the formation. The path of the virtual agent is centrally planned and the robots at the minima of the FPF are forced to move along with the virtual agent. In the neighborhood of obstacles, individual robots selfishly avoid collisions, thus marginally deviating from the formation. The proposed quasi-centralized approach introduces formation flexibility into the MRS, which enables MRS to effectively navigate in an obstacle-ridden workspace. Methodical analysis of the proposed approach and guidelines for selecting the FPF are presented. Results using a candidate FPF are shown that ensure a pentagonal formation effectively squeezes through a narrow passage avoiding any collisions with the walls.

With the rapid development of AI and robotics, transporting a large swarm of networked robots has foreseeable applications in the near future. Existing research in swarm robotics has mainly followed a bottom-up philosophy with predefined local coordination and control rules. However, it is arduous to verify the global requirements and analyze their performance. This motivates us to pursue a top-down approach, and develop a provable control strategy for deploying a robotic swarm to achieve a desired global configuration. Specifically, we use mean-field partial differential equations (PDEs) to model the swarm and control its mean-field density (i.e., probability density) over a bounded spatial domain using mean-field feedback. The presented control law uses density estimates as feedback signals and generates corresponding velocity fields that, by acting locally on individual robots, guide their global distribution to a target profile. The design of the velocity field is therefore centralized, but the implementation of the controller can be fully distributed -- individual robots sense the velocity field and derive their own velocity control signals accordingly. The key contribution lies in applying the concept of input-to-state stability (ISS) to show that the perturbed closed-loop system (a nonlinear and time-varying PDE) is locally ISS with respect to density estimation errors. The effectiveness of the proposed control laws is verified using agent-based simulations.

Machine Learning (ML) seems to be one of the most promising solution to automate partially or completely some of the complex tasks currently realized by humans, such as driving vehicles, recognizing voice, etc. It is also an opportunity to implement and embed new capabilities out of the reach of classical implementation techniques. However, ML techniques introduce new potential risks. Therefore, they have only been applied in systems where their benefits are considered worth the increase of risk. In practice, ML techniques raise multiple challenges that could prevent their use in systems submitted to certification constraints. But what are the actual challenges? Can they be overcome by selecting appropriate ML techniques, or by adopting new engineering or certification practices? These are some of the questions addressed by the ML Certification 3 Workgroup (WG) set-up by the Institut de Recherche Technologique Saint Exup\'ery de Toulouse (IRT), as part of the DEEL Project.

Along with the development of modern computing technology and social sciences, both theoretical research and practical applications of social computing have been continuously extended. In particular with the boom of artificial intelligence (AI), social computing is significantly influenced by AI. However, the conventional technologies of AI have drawbacks in dealing with more complicated and dynamic problems. Such deficiency can be rectified by hybrid human-artificial intelligence (H-AI) which integrates both human intelligence and AI into one unity, forming a new enhanced intelligence. H-AI in dealing with social problems shows the advantages that AI can not surpass. This paper firstly introduces the concept of H-AI. AI is the intelligence in the transition stage of H-AI, so the latest research progresses of AI in social computing are reviewed. Secondly, it summarizes typical challenges faced by AI in social computing, and makes it possible to introduce H-AI to solve these challenges. Finally, the paper proposes a holistic framework of social computing combining with H-AI, which consists of four layers: object layer, base layer, analysis layer, and application layer. It represents H-AI has significant advantages over AI in solving social problems.

Also, we organize different In this paper, we aim at providing a comprehensive works in this area based on their knowledge and capability outline of the different threads of work in human-levels and their teaming goal perspectives. Then, we highlight AI collaboration. By highlighting various aspects how recent works can be categorized regarding these of works on the human-AI team such as the flow dimensions. of complementing, task horizon, model representation, knowledge level, and teaming goal, we make a taxonomy of recent works according to these dimensions.