Reinforcement Learning is a popular research area. This is mainly because it aims to model systems that otherwise seem intractable. From the famous Atari paper by Deepmind, we have come far. An interesting avenue of study in reinforcement learning is that of communicating agents: a setup where agents can send messages to each other in order to cooperate. A good case where communication will be essential is that of an environment that is only partially observable to each agent, whereas more information is required for the agents to complete the task cooperatively.

A coalition of agents, or a single agent, has an ethical dilemma between several statements if each joint action of the coalition forces at least one specific statement among them to be true. For example, any action in the trolley dilemma forces one specific group of people to die. In many cases, agents face ethical dilemmas because they are restricted in the amount of the resources they are ready to sacrifice to overcome the dilemma. The paper presents a sound and complete modal logical system that describes properties of dilemmas for a given limit on a sacrifice.

When Salesforce introduced Einstein, its artificial intelligence platform in 2016, it was laying the ground work for artificial intelligence underpinnings across the platform. Since then the company has introduced a variety of AI enhancements to the Salesforce product family. Today, customer service got some AI updates. The goal of any customer service interaction is to get the customer answers as quickly as possible. Many users opt to use chat over phone, and Salesforce has added some AI features to help customer service agents get answers more quickly in the chat interface.

Robots are a key application for AI and in addition to an excellent plenary talk by Julie Shah of MIT, a whole track was dedicated to AI in robotics applications. Dan Kara, VP of robotics and intelligent systems for WTWH Media, outlined some of the challenges in building robots--not chatbots, he clarified, but robots that act in the physical world. "It seems like every year it's just around the corner," he said, but this year the tailwinds are picking up. Robotics is the foundation for much of our work thus far in artificial intelligence and machine learning, Kara argued. "It's only been fairly recently that you've started getting artificial intelligence or machine learning moving off into different labs," he said.

W e present an approach to generating natural language justifications of decisions derived from norm-based reasoning. Assuming an agent which maximally satisfies a set of rules specified in an object-oriented temporal logic, the user can ask factual questions (about the agent's rules, actions, and the extent to which the agent violated the rules) as well as "why" questions that require the agent comparing actual behavior to counterfactual trajectories with respect to these rules. To produce natural-sounding explanations, we focus on the subproblem of producing natural language clauses from statements in a fragment of temporal logic, and then describe how to embed these clauses into explanatory sentences. W e use a human judgment evaluation on a testbed task to compare our approach to variants in terms of intelligibility, mental model and perceived trust.

Many real-world applications require artificial agents to compete and coordinate with other agents in complex environments. As a stepping stone to this goal, the domain of StarCraft has emerged by consensus as an important challenge for artificial intelligence research, owing to its iconic and enduring status among the most difficult professional esports and its relevance to the real world in terms of its raw complexity and multiagent challenges. Over the course of a decade and numerous competitions 1–3, the best results have been made possible by hand-crafting major elements of the system, simplifying important aspects of the game, or using superhuman capabilities 4. Even with these modifications, no previous system has come close to rivalling the skill of top players in the full game. We chose to address the challenge of StarCraft using general purpose learning methods that are in principle applicable to other complex domains: a multi-agent reinforcement learning algorithm that uses data from both human and agent games within a diverse league of continually adapting strategies and counterstrategies, each represented by deep neural networks5,6. We evaluated our agent, AlphaStar, in the full game of StarCraft II, through a series of online games against human players. AlphaStar was rated at Grandmaster level for all three StarCraft races and above 99.8% of officially ranked human players.

For an autonomous embodied agent acting in the real world (e.g., an animal, a human, or a robot), perceptual categorization--the ability to make distinctions--is a hard problem (Harnad, 2005). First, based on the stimulation impinging on its sensory arrays (sensation) the agent has to rapidly determine and attend to what needs to be categorized. Second, the appearance and properties of objects or events in the environment being classified fluctuate continuously, for example owing to occlusions, or changes of distances and orientations with respect to the agent. And third, the environmental conditions (e.g., illumination, viewpoint, and background noise) vary considerably. There is much relevant work in computer vision that has been devoted to extracting scale- and translation-invariant low-level visual features and high-level multidimensional representations for the purpose of robust perceptual categorization (Riesenhuber & Poggio, 2002).

Single-agent deep reinforcement learning has achieved impressive performance in many domains, including playing Go [1, 2] and Atari games [3, 4]. However, many real world problems, such as traffic congestion reduction [5, 6], antenna tilt control [7], and dynamic resource allocation [8] are more naturally modeled as multi-agent systems. Unfortunately, directly deploying single-agent reinforcement learning to each agent in a multi-agent system does not result in satisfying performance [9, 10]. Particularly, in multi-agent reinforcement learning [8, 10-19], estimating the value function is challenging, because the environment is non-stationary from the perspective of an individual agent [10, 11]. To alleviate the issue, recently, multi-agent deep deterministic policy gradient (MADDPG) [10] proposed a centralized critic whose input is the concatenation of all agents' observations and actions.

Fairness is essential for human society, contributing to stability and productivity. Similarly, fairness is also the key for many multi-agent systems. Taking fairness into multi-agent learning could help multi-agent systems become both efficient and stable. However, learning efficiency and fairness simultaneously is a complex, multi-objective, joint-policy optimization. To tackle these difficulties, we propose FEN, a novel hierarchical reinforcement learning model. We first decompose fairness for each agent and propose fair-efficient reward that each agent learns its own policy to optimize. To avoid multi-objective conflict, we design a hierarchy consisting of a controller and several sub-policies, where the controller maximizes the fair-efficient reward by switching among the sub-policies that provides diverse behaviors to interact with the environment. FEN can be trained in a fully decentralized way, making it easy to be deployed in real-world applications. Empirically, we show that FEN easily learns both fairness and efficiency and significantly outperforms baselines in a variety of multi-agent scenarios.

Under appropriate cooperation protocols and parameter choices, fully decentralized solutions for stochastic optimization have been shown to match the performance of centralized solutions and result in linear speedup (in the number of agents) relative to non-cooperative approaches in the strongly-convex setting. More recently, these results have been extended to the pursuit of first-order stationary points in non-convex environments. In this work, we examine in detail the dependence of second-order convergence guarantees on the spectral properties of the combination policy for non-convex multi agent optimization. We establish linear speedup in saddle-point escape time in the number of agents for symmetric combination policies and study the potential for further improvement by employing asymmetric combination weights. The results imply that a linear speedup can be expected in the pursuit of second-order stationary points, which exclude local maxima as well as strict saddle-points and correspond to local or even global minima in many important learning settings.