In recent years, artificial intelligence (AI) tools, including natural language processing (NLP) techniques, have become increasingly sophisticated, achieving exceptional results in a variety of tasks. NLP techniques are specifically designed to understand human language and produce suitable responses, thus enabling communication between humans and artificial agents. Other studies also introduced goal-oriented agents that can autonomously navigate virtual or videogame environments. So far, NLP techniques and goal-oriented agents have typically been developed individually, rather than being combined into unified methods. Researchers at Georgia Institute of Technology and Facebook AI Research have recently explored the possibility of equipping goal-driven agents with NLP capabilities so that they can speak with other characters and complete desirable actions within fantasy game environments.

Effective communication is an important skill for enabling information exchange and cooperation in multi-agent settings. Indeed, emergent communication is now a vibrant field of research, with common settings involving discrete cheap-talk channels. One limitation of this setting is that it does not allow for the emergent protocols to generalize beyond the training partners. Furthermore, so far emergent communication has primarily focused on the use of symbolic channels. In this work, we extend this line of work to a new modality, by studying agents that learn to communicate via actuating their joints in a 3D environment. We show that under realistic assumptions, a non-uniform distribution of intents and a common-knowledge energy cost, these agents can find protocols that generalize to novel partners.

We provide a comprehensive analysis of the computational complexity of the outcome determination problem for the most important aggregation rules proposed in the literature on logic-based judgment aggregation. Judgment aggregation is a powerful and flexible framework for studying problems of collective decision making that has attracted interest in a range of disciplines, including Legal Theory, Philosophy, Economics, Political Science, and Artificial Intelligence. The problem of computing the outcome for a given list of individual judgments to be aggregated into a single collective judgment is the most fundamental algorithmic challenge arising in this context. Our analysis applies to several different variants of the basic framework of judgment aggregation that have been discussed in the literature, as well as to a new framework that encompasses all existing such frameworks in terms of expressive power and representational succinctness.

This paper introduces PettingZoo, a library of diverse sets of multi-agent environments under a single elegant Python API. PettingZoo was developed with the goal of accelerating research in multi-agent reinforcement learning, by creating a set of benchmark environments easily accessible to all researchers and a standardized API for the field. This goal is inspired by what OpenAI's Gym library did for accelerating research in single-agent reinforcement learning, and PettingZoo draws heavily from Gym in terms of API and user experience. PettingZoo is unique from other multi-agent environment libraries in that it's API is based on the model of Agent Environment Cycle ("AEC") games, which allows for the sensible representation of all varieties of games under one API for the first time. While retaining a very simple and Gym-like API, PettingZoo still allows access to low-level environment properties required by nontraditional learning methods. Reinforcement Learning ("RL") considers learning a policy -- a function that takes in an observation from an environment and emits an action -- that achieves the maximum expected discounted reward when acting in an environment, and it's capabilities have been one of the great success of modern machine learning. Multi-Agent Reinforcement Learning (MARL) in particular has been behind many of the most publicized achievements of modern machine learning -- AlphaGo Zero (Silver et al., 2017), OpenAI Five (OpenAI, 2018), AlphaStar (Vinyals et al., 2019) -- and has seen a boom in recent years.

Many complex manipulation tasks can be decomposed into smaller sub-tasks and distributed amongst multiple robotic arms working in parallel in a shared workspace. However, efficiently motion planning for such multi-arm systems remains a challenge due to its high degrees-of-freedom (DoF) and tightly coupled workspaces. While traditional centralized motion planner [1, 2, 3, 4, 5, 6] benefit from having access to all the information a motion planner module might need, these approaches fail to scale efficiently with the number of arms in the system (the team size) because their centralized components can become the bottleneck of the system. This scalability issue has limited multi-arm applications requiring large numbers of robotic arms operating in a tight workspace or in dynamic environments with moving targets. Less explored alternatives are decentralized motion planners, which treat the multi-arm system as a multi-agent system. Here, each arm is controlled by an agent that receives as input a partial observation of the system's state and computes a motion plan for only itself. Naturally, decentralized motion planners scale efficiently, but designing such a controller for a task as complex as generic multi-arm motion planning remains a challenge. Through observations of other arms' states alone, decentralized motion planners must efficiently coordinate to avoid collisions and cooperate to collectively reach their target end-effector poses, all the while having control over only its arm (Figure 1). An ideal candidate for a multi-arm motion planner should have the following characteristics: - Scalability: The runtime should scale efficiently with the number of arms in the system.

Autonomous robot-assisted feeding requires the ability to acquire a wide variety of food items. However, it is impossible for such a system to be trained on all types of food in existence. Therefore, a key challenge is choosing a manipulation strategy for a previously unseen food item. Previous work showed that the problem can be represented as a linear contextual bandit on visual information. However, food has a wide variety of multi-modal properties relevant to manipulation that can be hard to distinguish visually. Our key insight is that we can leverage the haptic information we collect during manipulation to learn some of these properties and more quickly adapt our visual model to previously unseen food. In general, we propose a modified linear contextual bandit framework augmented with post hoc context observed after action selection to empirically increase learning speed (as measured by cross-validation mean square error) and reduce cumulative regret. Experiments on synthetic data demonstrate that this effect is more pronounced when the dimensionality of the context is large relative to the post hoc context or when the post hoc context model is particularly easy to learn. Finally, we apply this framework to the bite acquisition problem and demonstrate the acquisition of 8 previously unseen types of food with 21% fewer failures across 64 attempts.

This paper presents the computational details of our emotion model, EEGS, and also provides an overview of a three-stage validation methodology used for the evaluation of our model, which can also be applicable for other computational models of emotion. A major gap in existing emotion modelling literature has been the lack of computational/technical details of the implemented models, which not only makes it difficult for early-stage researchers to understand the area but also prevents benchmarking of the developed models for expert researchers. We partly addressed these issues by presenting technical details for the computation of appraisal variables in our previous work. In this paper, we present mathematical formulas for the calculation of emotion intensities based on the theoretical premises of appraisal theory. Moreover, we will discuss how we enable our emotion model to reach to a regulated emotional state for social acceptability of autonomous agents. We hope this paper will allow a better transparency of knowledge, accurate benchmarking and further evolution of the field of emotion modelling.

Everything Artificial Intelligence has ever been, hopes to be, or currently is to the enterprise has been encapsulated in a single emergent concept, a hybrid term, simultaneously detailing exactly where it is today, and just where it's headed in the coming year. The ModelOps notion is so emblematic of AI because it gives credence to its full breadth (from machine learning to its knowledge base), which Gartner indicates involves rules, agents, knowledge graphs, and more. ModelOps is about more than simply operationalizing and governing AI models. Moreover, it involves doing so onsite while leveraging the advantages of the cloud and, when it comes to AI's machine learning prowess, with a range of approaches rooted in supervised, unsupervised, and even reinforcement learning. Implicit to these capabilities is the need to position machine learning models at the edge, supersede their traditional training data limitations (and methods), and imbibe everything from streaming to static data for a predictive exactness based on the most current data possible.

Reinforcement learning models are trained, using a similar concept by animal researchers to train animals. For a very long period, artificial intelligence agents were trained on machine learning models to perform tasks that are usually done by humans. The neural networks of machine learning models are designed and trained in such a format that they perform the tasks without any human intervention or supervision. However, ever since its inception, the researchers and scientists are curious to induce cognitive abilities into artificial intelligence agents. For a decade, despite the experiments designed to train the artificial neural network by utilizing the human cognitive ability for adopting common sense, the researchers were unable to reach into a reasonable conclusion. The researchers were resorting to behavioral science and neuroscience earlier to induce common sense into the artificial intelligence agents.

Joining multiple decision-makers together is a powerful way to obtain more sophisticated decision-making systems, but requires to address the questions of division of labor and specialization. We investigate in how far information constraints in hierarchies of experts not only provide a principled method for regularization but also to enforce specialization. In particular, we devise an information-theoretically motivated on-line learning rule that allows partitioning of the problem space into multiple sub-problems that can be solved by the individual experts. We demonstrate two different ways to apply our method: (i) partitioning problems based on individual data samples and (ii) based on sets of data samples representing tasks. Approach (i) equips the system with the ability to solve complex decision-making problems by finding an optimal combination of local expert decision-makers. Approach (ii) leads to decision-makers specialized in solving families of tasks, which equips the system with the ability to solve meta-learning problems. We show the broad applicability of our approach on a range of problems including classification, regression, density estimation, and reinforcement learning problems, both in the standard machine learning setup and in a meta-learning setting.