Problems that claim several agents to find no-conflicts paths from their start locations to their destinations are named as cooperative pathfinding problems. This problem can be efficiently solved by the Multi-agent RRT*(MA-RRT*) algorithm, which offers better scalability than some traditional algorithms, such as Optimal Anytime(OA), in sparse environments. However, MA-RRT* cannot effectively find solutions in relatively dense environments, cause some random samples in the free space cannot be explored by the rapidly random tree, which hinders the application of MA-RRT* in a more complicated real-world. This paper proposes an improved version of MA-RRT *, called Multi-agent RRT* Potential Field (MA-RRT*PF), an anytime algorithm that can efficiently guide the rapidly random tree to the free space in relatively dense environments. It works by incorporating a potential field to the GREEDY function to enhance the ability to avoid the obstacles. The results show that MA-RRT*PF performs much better than MA-RRT* in relatively dense environments in terms of scalability while still maintaining the solution quality.

Many real-world multi-agent reinforcement learning applications require agents to communicate, assisted by a communication protocol. These applications face a common and critical issue of communication's limited bandwidth that constrains agents' ability to cooperate successfully. In this paper, rather than proposing a fixed communication protocol, we develop an Informative Multi-Agent Communication (IMAC) method to learn efficient communication protocols. Our contributions are threefold. First, we notice a fact that a limited bandwidth translates into a constraint on the communicated message entropy, thus paving the way of controlling the bandwidth. Second, we introduce a customized batch-norm layer, which controls the messages' entropy to simulate the limited bandwidth constraint. Third, we apply the information bottleneck method to discover the optimal communication protocol, which can satisfy a bandwidth constraint via training with the prior distribution in the method. To demonstrate the efficacy of our method, we conduct extensive experiments in various cooperative and competitive multi-agent tasks across two dimensions: the number of agents and different bandwidths. We show that IMAC converges fast, and leads to efficient communication among agents under the limited-bandwidth constraint as compared to many baseline methods.

In cooperative pathfinding problems, no-conflicts paths that bring several agents from their start location to their destination need to be planned. This problem can be efficiently solved by Multi-agent RRT*(MA-RRT*) algorithm, which offers better scalability than the classical algorithms, such as Optimal Anytime(OA), in sparse environments. However, the implementation of this algorithm in systems with limited memory is hindered because the number of nodes in the tree grows indefinitely as the paths get optimized. This paper proposes an improved version of MA-RRT*, called Multi-agent RRT* Fixed Node(MA-RRT*FN), which limits the number of nodes stored in the tree by removing the weak nodes which are not likely on the path reaching the goal. The results show that MA-RRT*FN performs close to MA-RRT* in terms of scalability and solution quality while the memory required is much lower and fixed.

Google Cloud is making Contact Center AI generally available for use today. The cloud service is built with conversational AI engine Dialogflow to automate interactions with customers in call centers. Contact Center AI contains Virtual Agent to automatically respond to customer queries with voice or text or handoff the conversation to a person when a bot is unable to help a customer. Agent Assist uses natural language processing to augment customer service agent interactions with customers when a bot is unable to help a customer. The news comes today as Google pushed its rich communication services (RCS) to Android Messages users in the United States, and days after Google's experimental unit Area 120 CallJoy service for answering phone calls and customer questions for small businesses got an upgrade.

Google Cloud is making Contact Center AI generally available for use today. The cloud service is built with Dialogflow of the conversational AI engine to automate interactions with customers in call centers. Contact Center AI contains Virtual Agent to automatically respond to customer inquiries with voice or text or transfer the conversation to a person when a bot cannot help a client. Agent Assist uses natural language processing to increase customer service agent interactions with customers when a bot cannot help a client. The news comes today when Google launched its rich communication services (RCS) to Android message users in the United States, and days after Google's experimental unit, the Area 120 CallJoy service to answer phone calls and questions Small business customers get an update.

A famous television series kicked off with its first episode titled "Winter is Coming." This phrase ended up carrying more weight than just the arrival of a season in the fictional series. In real life, some might say another monumental event is nearly upon us: Black Friday. References to Black Friday began as early as the 1950's in the United States. It wasn't until the 1980's that it came to refer to the retail shopping period following the Thanksgiving holiday, with one explanation suggesting the color indicated this being the time at which retail companies moved from operating at a loss (or "in the red") to profitability ("in the black").

Google has announced a notable upgrade to CallJoy, the automated, cloud-based telephone customer service agent first debuted back in May. CallJoy is the handiwork of Area 120, Google's "experimental" off-shoot, so it's not technically a full-fledged Google product yet. But the service is chargeable and available to all businesses across the U.S., so for all intents and purposes this really is an official Google SMB product. Over the past four months, businesses that signed up to CallJoy for $39 per month (per location) were given a virtual phone number that is capable of blocking spam calls; providing basic business information such as opening hours; and serving up analytics, including call recordings with automated transcriptions of each call. The automated agent could also send the caller an SMS link to book an appointment or place an order.

The Pommerman simulation was recently developed to mimic the classic Japanese game Bomberman, and focuses on competitive gameplay in a multi-agent setting. We focus on the 2$\times$2 team version of Pommerman, developed for a competition at NeurIPS 2018. Our methodology involves training an agent initially through imitation learning on a noisy expert policy, followed by a proximal-policy optimization (PPO) reinforcement learning algorithm. The basic PPO approach is modified for stable transition from the imitation learning phase through reward shaping, action filters based on heuristics, and curriculum learning. The proposed methodology is able to beat heuristic and pure reinforcement learning baselines with a combined 100,000 training games, significantly faster than other non-tree-search methods in literature. We present results against multiple agents provided by the developers of the simulation, including some that we have enhanced. We include a sensitivity analysis over different parameters, and highlight undesirable effects of some strategies that initially appear promising. Since Pommerman is a complex multi-agent competitive environment, the strategies developed here provide insights into several real-world problems with characteristics such as partial observability, decentralized execution (without communication), and very sparse and delayed rewards.

We consider the issue of multiple agents learning to communicate through reinforcement learning within partially observable environments, with a focus on information asymmetry in the second part of our work. We provide a review of the recent algorithms developed to improve the agents' policy by allowing the sharing of information between agents and the learning of communication strategies, with a focus on Deep Recurrent Q-Network-based models. We also describe recent efforts to interpret the languages generated by these agents and study their properties in an attempt to generate human-language-like sentences. We discuss the metrics used to evaluate the generated communication strategies and propose a novel entropy-based evaluation metric. Finally, we address the issue of the cost of communication and introduce the idea of an experimental setup to expose this cost in cooperative-competitive game.

De-An Huang 1, 2, Y u-Wei Chao, 2, Chris Paxton, 2, Xinke Deng 2, 3, Li Fei-Fei 1, Juan Carlos Niebles 1, Animesh Garg 2, 4, Dieter Fox 2, 5 Abstract -- We address goal-based imitation learning, where the aim is to output the symbolic goal from a third-person video demonstration. This enables the robot to plan for execution and reproduce the same goal in a completely different environment. The key challenge is that the goal of a video demonstration is often ambiguous at the level of semantic actions. The human demonstrators might unintentionally achieve certain subgoals in the demonstrations with their actions. Our main contribution is to propose a motion reasoning framework that combines task and motion planning to disambiguate the true intention of the demonstrator in the video demonstration. This allows us to robustly recognize the goals that cannot be disambiguated by previous action-based approaches. We evaluate our approach by collecting a dataset of 96 video demonstrations in a mockup kitchen environment. We show that our motion reasoning plays an important role in recognizing the actual goal of the demonstrator and improves the success rate by over 20%. We further show that by using the automatically inferred goal from the video demonstration, our robot is able to reproduce the same task in a real kitchen environment. I NTRODUCTION We are interested in allowing robots to learn new tasks from video demonstrations. Recently, there has been rapid progress in imitation learning [1-4], which even enables learning a new task from a single demonstration of the task [5-7]. By leveraging meta-learning [8], the robot learns to follow the actions in the demonstration.