Cooperation of a large number of possibly small-sized robots, called robotic swarm, will play a significant role in complex missions that existing operational concepts using a few large robots could not deal with [1]. Even if every single robot (or called agent) in a swarm is incapable of accomplishing a task alone, their cooperation will lead to successful outcomes [2]-[5]. The possible applications include environmental monitoring [6], ad-hoc network relay [7], disaster management [8], cooperative radar jamming [9], to name a few. Due to the large cardinality of a swarm robot system, however, it is infeasible for human operators to supervise each agent directly, but needed to entrust the swarm with certain levels of decision-makings (e.g., task allocation, path planning, and individual control). Thereby, what only remains is to provide a high-level mission description, which is manageable for a few or even a single human operator. Nevertheless, there still exist various challenges in the autonomous decisionmaking of robotic swarms. Among them, this paper addresses a task allocation problem where the number of agents is higher than that of tasks: how to partition a set of agents into subgroups and assign the subgroups to each task.

The U.S Department of Homeland Security had funded a research approximately 6 years ago of the virtual border agent technology, better known as AVATAR (Automated Virtual Agent for Truth Assessments in Real Time) and had tested it at the U.S Mexico border on travelers voluntarily. Canada and EU has also tested the robot like kiosk that is asking travelers a series of questions. If the trend continues, International Travelers could be speaking with kiosk to determine if they are lying on any aspect at an airport or border crossings. The technology can also be used to screen the refugees and unwanted travelers travelling to any country. It can also be used to screen the citizenship applications, processing visas and many other such inter-related services.

Robotic systems, working together as a team, are becoming valuable players in different real-world applications, from disaster response to warehouse fulfillment services. Centralized solutions for coordinating multi-robot teams often suffer from poor scalability and vulnerability to communication disruptions. This paper develops a decentralized multi-agent task allocation (Dec-MATA) algorithm for multi-robot applications. The task planning problem is posed as a maximum-weighted matching of a bipartite graph, the solution of which using the blossom algorithm allows each robot to autonomously identify the optimal sequence of tasks it should undertake. The graph weights are determined based on a soft clustering process, which also plays a problem decomposition role seeking to reduce the complexity of the individual-agents' task assignment problems. To evaluate the new Dec-MATA algorithm, a series of case studies (of varying complexity) are performed, with tasks being distributed randomly over an observable 2D environment. A centralized approach, based on a state-of-the-art MILP formulation of the multi-Traveling Salesman problem is used for comparative analysis. While getting within 7-28% of the optimal cost obtained by the centralized algorithm, the Dec-MATA algorithm is found to be 1-3 orders of magnitude faster and minimally sensitive to task-to-robot ratios, unlike the centralized algorithm.

Designing hierarchical reinforcement learning algorithms that induce a notion of safety is not only vital for safety-critical applications, but also, brings better understanding of an artificially intelligent agent's decisions. While learning end-to-end options automatically has been fully realized recently, we propose a solution to learning safe options. We introduce the idea of controllability of states based on the temporal difference errors in the option-critic framework. We then derive the policy-gradient theorem with controllability and propose a novel framework called safe option-critic. We demonstrate the effectiveness of our approach in the four-rooms grid-world, cartpole, and three games in the Arcade Learning Environment (ALE): MsPacman, Amidar and Q*Bert. Learning of end-to-end options with the proposed notion of safety achieves reduction in the variance of return and boosts the performance in environments with intrinsic variability in the reward structure. More importantly, the proposed algorithm outperforms the vanilla options in all the environments and primitive actions in two out of three ALE games.

Active systems can produce a far greater variety of ordered patterns than conventional equilibrium systems. In particular, transitions between disorder and either polar- or nematically ordered phases have been predicted and observed in two-dimensional active systems. However, coexistence between phases of different types of order has not been reported. We demonstrate the emergence of dynamic coexistence of ordered states with fluctuating nematic and polar symmetry in an actomyosin motility assay. Combining experiments with agent-based simulations, we identify sufficiently weak interactions that lack a clear alignment symmetry as a prerequisite for coexistence. Thus, the symmetry of macroscopic order becomes an emergent and dynamic property of the active system.

I talked to Dirk Erlacher, the CEO of Agilox, on this topic. Austrian headquartered Agilox designs and manufactures mobile logistics robots that use "swarm intelligence" to intelligently navigate through warehouses and factories, delivering pallets and totes where they are needed. A mobile logistics robot (MLR) is a more advanced form of an automatic guided vehicle (AGV); AGVs are used to reduce labor by taking over tasks that were traditionally performed with fork lifts. More complex AGVs have fleet management software. This software makes sure that not too many AGVs are in the same aisles, decides which AGV has the right of way at crossings, and in more complex scenarios, decides which unit will be used to complete a particular task and how it will navigate through the facility.

Most artificial intelligence(AI) systems nowadays are based on a single agent tackling a task or, in the case of adversarial models, a couple of agents that compete against each other to improve the overall behavior of a system. However, many cognition problems in the real world are the result of knowledge built by large groups of people. Take for example a self-driving car scenario, the decisions of any agent are the result of the behavior of many other agents in the scenario. Many scenarios in financial markets or economics are also the result of coordinated actions between large groups of entities. How can we mimic that behavior in artificial intelligence(AI) agents?

The idea of reusing information from previously learned tasks (source tasks) for the learning of new tasks (target tasks) has the potential to significantly improve the sample efficiency reinforcement learning agents. In this work, we describe an approach to concisely store and represent learned task knowledge, and reuse it by allowing it to guide the exploration of an agent while it learns new tasks. In order to do so, we use a measure of similarity that is defined directly in the space of parameterized representations of the value functions. This similarity measure is also used as a basis for a variant of the growing self-organizing map algorithm, which is simultaneously used to enable the storage of previously acquired task knowledge in an adaptive and scalable manner. We empirically validate our approach in a simulated navigation environment and discuss possible extensions to this approach along with potential applications where it could be particularly useful.

A long-standing problem in model free reinforcement learning (RL) is that it requires a large number of trials to learn a good policy, especially in environments with sparse rewards. We explore a method to increase the sample efficiency of RL when we have access to demonstrations. Our approach, which we call Backplay, uses a single demonstration to construct a curriculum for a given task. Rather than starting each training episode in the environment's fixed initial state, we start the agent near the end of the demonstration and move the starting point backwards during the course of training until we reach the initial state. We perform experiments in a competitive four player game (Pommerman) and a path-finding maze game. We find that this weak form of guidance provides significant gains in sample complexity with a stark advantage in sparse reward environments. In some cases, standard RL did not yield any improvement while Backplay reached success rates greater than 50% and generalized to unseen initial conditions in the same amount of training time. Additionally, we see that agents trained via Backplay can learn policies superior to those of the original demonstration.

In an economic market, sellers, infomediaries and customers constitute an economic network. Each seller has her own customer group and the seller's private customers are unobservable to other sellers. Therefore, a seller can only sell commodities among her own customers unless other sellers or infomediaries share her sale information to their customer groups. However, a seller is not incentivized to share others' sale information by default, which leads to inefficient resource allocation and limited revenue for the sale. To tackle this problem, we develop a novel mechanism called customer sharing mechanism (CSM) which incentivizes all sellers to share each other's sale information to their private customer groups. Furthermore, CSM also incentivizes all customers to truthfully participate in the sale.