Efficiently discovering relevant Web services with respect to a specific user query has become a growing challenge owing to the incredible growth in the field of web technologies. In previous works, different clustering models have been used to address these issues. But, most of the traditional clustering techniques are computationally intensive and fail to address all the problems involved. Also, the current standards fail to incorporate the semantic relatedness of Web services during clustering and retrieval resulting in decreased performance. In this paper, we propose a framework for web services retrieval that uses a bottom-up, decentralized and self organising approach to cluster available services. It also provides online, dynamic computation of clusters thus overcoming the drawbacks of traditional clustering methods. We also use the semantic similarity between Web services for the clustering process to enhance the precision and lower the recall.

We present SLOT-V, a novel supervised learning framework that learns observer models (human preferences) from robot motion trajectories in a legibility context. Legibility measures how easily a (human) observer can infer the robot's goal from a robot motion trajectory. When generating such trajectories, existing planners often rely on an observer model that estimates the quality of trajectory candidates. These observer models are frequently hand-crafted or, occasionally, learned from demonstrations. Here, we propose to learn them in a supervised manner using the same data format that is frequently used during the evaluation of aforementioned approaches. We then demonstrate the generality of SLOT-V using a Franka Emika in a simulated manipulation environment. For this, we show that it can learn to closely predict various hand-crafted observer models, i.e., that SLOT-V's hypothesis space encompasses existing handcrafted models. Next, we showcase SLOT-V's ability to generalize by showing that a trained model continues to perform well in environments with unseen goal configurations and/or goal counts. Finally, we benchmark SLOT-V's sample efficiency (and performance) against an existing IRL approach and show that SLOT-V learns better observer models with less data. Combined, these results suggest that SLOT-V can learn viable observer models. Better observer models imply more legible trajectories, which may - in turn - lead to better and more transparent human-robot interaction.

In the following article I discuss how to generate emergent behavior in AI characters using Unreal Engine, Reinforcement Learning, and the free machine learning plugin MindMaker. The aim is that the interested reader can use this as a guide for creating emergent behavior in their own game project or embodied AI character. Emergent behavior refers to behaviors that are not pre-programmed but develop organically in response to some environmental stimuli. Emergent behavior is common to many if not all forms of life, being a function of evolution itself. It is also more recently a feature of embodied artificial agents. When one employs emergent behavior methods, one does not rigidly program specific actions for the AI, but instead allows them to "evolve" through some adaptive algorithm such as genetic programming, reinforcement learning, or Monte Carlo methods.

Multi-Agent Reinforcement Learning (MARL) -- where multiple agents learn to interact in a shared dynamic environment -- permeates across a wide range of critical applications. While there has been substantial progress on understanding the global convergence of policy optimization methods in single-agent RL, designing and analysis of efficient policy optimization algorithms in the MARL setting present significant challenges, which unfortunately, remain highly inadequately addressed by existing theory. In this paper, we focus on the most basic setting of competitive multi-agent RL, namely two-player zero-sum Markov games, and study equilibrium finding algorithms in both the infinite-horizon discounted setting and the finite-horizon episodic setting. We propose a single-loop policy optimization method with symmetric updates from both agents, where the policy is updated via the entropy-regularized optimistic multiplicative weights update (OMWU) method and the value is updated on a slower timescale. We show that, in the full-information tabular setting, the proposed method achieves a finite-time last-iterate linear convergence to the quantal response equilibrium of the regularized problem, which translates to a sublinear last-iterate convergence to the Nash equilibrium by controlling the amount of regularization. Our convergence results improve upon the best known iteration complexities, and lead to a better understanding of policy optimization in competitive Markov games.

Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these systems can be made cognizant of the context in which communication exchange takes place, providing avenues for novel design insights. This tutorial summarizes the efforts to date, starting from its early adaptations, semantic-aware and task-oriented communications, covering the foundations, algorithms and potential implementations. The focus is on approaches that utilize information theory to provide the foundations, as well as the significant role of learning in semantics and task-aware communications.

In this paper, we introduce a new class of potential fields, i.e., meta navigation functions (MNFs) to coordinate multi-agent systems. Thanks to the MNF formulation, agents can contribute to each other's coordination via partial and/or total associations, contrary to traditional decentralized navigation functions (DNFs). In particular, agents may stimulate each other via their MNFs. Moreover, MNFs need to be confined which is a weaker condition compared to the Morse condition of DNFs. An MNF is composed of a confined function and an attraction kernel. The critical points of the former can be confined in a safe region around a target critical point. The collision-free trajectory of an agent and its associations to its peers are governed by a confined function before reaching its safe region. Then, the attraction kernel drives the agent to its target in the safe region. MNFs provide faster coordination compared to DNFs. We illustrate how MNFs may exhibit some social behaviors in the course of partial and total associations among agents. Our simulations verify the efficiency of MNFs to coordinate complex swarms of agents.

In transportation networks, where traffic lights have traditionally been used for vehicle coordination, intersections act as natural bottlenecks. A formidable challenge for existing automated intersections lies in detecting and reasoning about uncertainty from the operating environment and human-driven vehicles. In this paper, we propose a risk-aware intelligent intersection system for autonomous vehicles (AVs) as well as human-driven vehicles (HVs). We cast the problem as a novel class of Multi-agent Chance-Constrained Stochastic Shortest Path (MCC-SSP) problems and devise an exact Integer Linear Programming (ILP) formulation that is scalable in the number of agents' interaction points (e.g., potential collision points at the intersection). In particular, when the number of agents within an interaction point is small, which is often the case in intersections, the ILP has a polynomial number of variables and constraints. To further improve the running time performance, we show that the collision risk computation can be performed offline. Additionally, a trajectory optimization workflow is provided to generate risk-aware trajectories for any given intersection. The proposed framework is implemented in CARLA simulator and evaluated under a fully autonomous intersection with AVs only as well as in a hybrid setup with a signalized intersection for HVs and an intelligent scheme for AVs. As verified via simulations, the featured approach improves intersection's efficiency by up to $200\%$ while also conforming to the specified tunable risk threshold.

We develop a novel formulation of the Performance Estimation Problem (PEP) for decentralized optimization whose size is independent of the number of agents in the network. The PEP approach allows computing automatically the worst-case performance and worst-case instance of first-order optimization methods by solving an SDP. Unlike previous work, the size of our new PEP formulation is independent of the network size. For this purpose, we take a global view of the decentralized problem and we also decouple the consensus subspace and its orthogonal complement. We apply our methodology to different decentralized methods such as DGD, DIGing and EXTRA and obtain numerically tight performance guarantees that are valid for any network size.

ABSTRACT With the emerging connected-vehicle technologies and smart roads, the need for intelligent adaptive traffic signal controls is more than ever before. This paper proposes a novel Economicdriven Adaptive Traffic Signal Control (eATSC) model with a hyper control variable - interest rate defined in economics for traffic signal control at signalized intersections. The eATSC uses a continuous compounding function that captures both the total number of vehicles and the accumulated waiting time of each vehicle to compute penalties for different directions. The computed penalties grow with waiting time and is used for signal control decisions. Each intersection is assigned two intelligent agents adjusting interest rate and signal length for different directions according to the traffic patterns, respectively. The problem is formulated as a Markov Decision Process (MDP) problem to reduce congestions, and a two-agent Double Dueling Deep Q Network (DDDQN) is utilized to solve the problem. Under the optimal policy, the agents can select the optimal interest rates and signal time to minimize the likelihood of traffic congestions. To evaluate the superiority of our method, a VISSIM simulation model with classic four-leg signalized intersections is developed. The results indicate that the proposed model is adequately able to maintain healthy traffic flow at the intersection. INTRODUCTION Many studies have shown that adaptive signal control (ASC) improves traffic performance, such as emissions, travel time, and fuel consumption by at least 10% [1].

Under RoboCup rules, each team must consist of fully autonomous robot footballers; once the match begins, the human coaches are not allowed to interact with their robot players. Stone, who is also a professor of computer science at the University of Texas at Austin in the United States and the director of the university's Learning Agents Research Group, noted that achievements in robotic football have advanced a range of AI and machine learning fields. This includes human-robot interaction, computer vision and multiagent systems, among others.