In this paper, we derive a new model of synaptic plasticity, b ased on recent algorithms for reinforcement learning (in which an age nt attempts to learn appropriate actions to maximize its long-term averag e reward). We show that these direct reinforcement learning algorithms a lso give locally optimal performance for the problem of reinforcement learn ing with multiple agents, without any explicit communication between a gents. By considering a network of spiking neurons as a collection of agen ts attempting to maximize the long-term average of a reward signal, we deri ve a synaptic update rule that is qualitatively similar to Hebb's post ulate. This rule requires only simple computations, such as addition and lea ky integration, and involves only quantities that are available in the vicin ity of the synapse. Furthermore, it leads to synaptic connection strengths tha t give locally optimal values of the long term average reward. The reinforcem ent learning paradigm is sufficiently broad to encompass many learning pr oblems that are solved by the brain. We illustrate, with simulations, th at the approach is effective for simple pattern classification and motor learn ing tasks. It is widely accepted that the functions performed by neural circuits are modified by adjustments to the strength of the synaptic connectio ns between neurons. 1 In the 1940s, Donald Hebb speculated that such adjustments a re associated with simultaneous (or nearly simultaneous) firing of the presyna ptic and postsynaptic neurons [14]: When an axon of cell A ... persistently takes part in firing [cell B ], some growth process or metabolic change takes place [to incr ease] A's efficacy as one of the cells firing B .

The IKEA Furniture Assembly Environment is one of the first benchmarks for testing and accelerating the automation of complex manipulation tasks. The environment is designed to advance reinforcement learning from simple toy tasks to complex tasks requiring both long-term planning and sophisticated low-level control. Our environment supports over 80 different furniture models, Sawyer and Baxter robot simulation, and domain randomization. The IKEA Furniture Assembly Environment is a testbed for methods aiming to solve complex manipulation tasks. The environment is publicly available at https://clvrai.com/furniture

Problems that claim several agents to find no-conflicts paths from their start locations to their destinations are named as cooperative pathfinding problems. This problem can be efficiently solved by the Multi-agent RRT*(MA-RRT*) algorithm, which offers better scalability than some traditional algorithms, such as Optimal Anytime(OA), in sparse environments. However, MA-RRT* cannot effectively find solutions in relatively dense environments, cause some random samples in the free space cannot be explored by the rapidly random tree, which hinders the application of MA-RRT* in a more complicated real-world. This paper proposes an improved version of MA-RRT *, called Multi-agent RRT* Potential Field (MA-RRT*PF), an anytime algorithm that can efficiently guide the rapidly random tree to the free space in relatively dense environments. It works by incorporating a potential field to the GREEDY function to enhance the ability to avoid the obstacles. The results show that MA-RRT*PF performs much better than MA-RRT* in relatively dense environments in terms of scalability while still maintaining the solution quality.

Many real-world multi-agent reinforcement learning applications require agents to communicate, assisted by a communication protocol. These applications face a common and critical issue of communication's limited bandwidth that constrains agents' ability to cooperate successfully. In this paper, rather than proposing a fixed communication protocol, we develop an Informative Multi-Agent Communication (IMAC) method to learn efficient communication protocols. Our contributions are threefold. First, we notice a fact that a limited bandwidth translates into a constraint on the communicated message entropy, thus paving the way of controlling the bandwidth. Second, we introduce a customized batch-norm layer, which controls the messages' entropy to simulate the limited bandwidth constraint. Third, we apply the information bottleneck method to discover the optimal communication protocol, which can satisfy a bandwidth constraint via training with the prior distribution in the method. To demonstrate the efficacy of our method, we conduct extensive experiments in various cooperative and competitive multi-agent tasks across two dimensions: the number of agents and different bandwidths. We show that IMAC converges fast, and leads to efficient communication among agents under the limited-bandwidth constraint as compared to many baseline methods.

In cooperative pathfinding problems, no-conflicts paths that bring several agents from their start location to their destination need to be planned. This problem can be efficiently solved by Multi-agent RRT*(MA-RRT*) algorithm, which offers better scalability than the classical algorithms, such as Optimal Anytime(OA), in sparse environments. However, the implementation of this algorithm in systems with limited memory is hindered because the number of nodes in the tree grows indefinitely as the paths get optimized. This paper proposes an improved version of MA-RRT*, called Multi-agent RRT* Fixed Node(MA-RRT*FN), which limits the number of nodes stored in the tree by removing the weak nodes which are not likely on the path reaching the goal. The results show that MA-RRT*FN performs close to MA-RRT* in terms of scalability and solution quality while the memory required is much lower and fixed.

Google Cloud is making Contact Center AI generally available for use today. The cloud service is built with conversational AI engine Dialogflow to automate interactions with customers in call centers. Contact Center AI contains Virtual Agent to automatically respond to customer queries with voice or text or handoff the conversation to a person when a bot is unable to help a customer. Agent Assist uses natural language processing to augment customer service agent interactions with customers when a bot is unable to help a customer. The news comes today as Google pushed its rich communication services (RCS) to Android Messages users in the United States, and days after Google's experimental unit Area 120 CallJoy service for answering phone calls and customer questions for small businesses got an upgrade.

Google Cloud is making Contact Center AI generally available for use today. The cloud service is built with Dialogflow of the conversational AI engine to automate interactions with customers in call centers. Contact Center AI contains Virtual Agent to automatically respond to customer inquiries with voice or text or transfer the conversation to a person when a bot cannot help a client. Agent Assist uses natural language processing to increase customer service agent interactions with customers when a bot cannot help a client. The news comes today when Google launched its rich communication services (RCS) to Android message users in the United States, and days after Google's experimental unit, the Area 120 CallJoy service to answer phone calls and questions Small business customers get an update.

A famous television series kicked off with its first episode titled "Winter is Coming." This phrase ended up carrying more weight than just the arrival of a season in the fictional series. In real life, some might say another monumental event is nearly upon us: Black Friday. References to Black Friday began as early as the 1950's in the United States. It wasn't until the 1980's that it came to refer to the retail shopping period following the Thanksgiving holiday, with one explanation suggesting the color indicated this being the time at which retail companies moved from operating at a loss (or "in the red") to profitability ("in the black").

Google has announced a notable upgrade to CallJoy, the automated, cloud-based telephone customer service agent first debuted back in May. CallJoy is the handiwork of Area 120, Google's "experimental" off-shoot, so it's not technically a full-fledged Google product yet. But the service is chargeable and available to all businesses across the U.S., so for all intents and purposes this really is an official Google SMB product. Over the past four months, businesses that signed up to CallJoy for $39 per month (per location) were given a virtual phone number that is capable of blocking spam calls; providing basic business information such as opening hours; and serving up analytics, including call recordings with automated transcriptions of each call. The automated agent could also send the caller an SMS link to book an appointment or place an order.

The Pommerman simulation was recently developed to mimic the classic Japanese game Bomberman, and focuses on competitive gameplay in a multi-agent setting. We focus on the 2$\times$2 team version of Pommerman, developed for a competition at NeurIPS 2018. Our methodology involves training an agent initially through imitation learning on a noisy expert policy, followed by a proximal-policy optimization (PPO) reinforcement learning algorithm. The basic PPO approach is modified for stable transition from the imitation learning phase through reward shaping, action filters based on heuristics, and curriculum learning. The proposed methodology is able to beat heuristic and pure reinforcement learning baselines with a combined 100,000 training games, significantly faster than other non-tree-search methods in literature. We present results against multiple agents provided by the developers of the simulation, including some that we have enhanced. We include a sensitivity analysis over different parameters, and highlight undesirable effects of some strategies that initially appear promising. Since Pommerman is a complex multi-agent competitive environment, the strategies developed here provide insights into several real-world problems with characteristics such as partial observability, decentralized execution (without communication), and very sparse and delayed rewards.