Today's information systems are complex, distributed, and need to scale over millions of users and a variety of devices, with guaranteed uptimes. As a result, top-down approaches for systems design and engineering are becoming increasingly infeasible. Starting sometime in the 1990s, a branch of systems engineering, has approached the problem of systemic complexity in a bottom-up fashion, by designing "autonomous" or "intelligent" agents that can proactively and autonomously act and decide on their own-to address specific, local issues pertaining to their immediate requirements. They also can communicate and coordinate with one another to jointly solve larger problems. The autonomous nature of agents require some form of a rationale that justifies their actions. Given that, objectoriented modeling had attracted mainstream attention at that time, the distinction between mechanistic "objects" and autonomous "agents" were often summarized with this slogan (Jennings et al., 1998): Objects do it for free, agents do it for money.

With the emerging technologies in Intelligent Transportation System (ITS), the adaptive operation of road space is likely to be realised within decades. An intelligent street can learn and improve its decision-making on the right-of-way (ROW) for road users, liberating more active pedestrian space while maintaining traffic safety and efficiency. However, there is a lack of effective controlling techniques for these adaptive street infrastructures. To fill this gap in existing studies, we formulate this control problem as a Markov Game and develop a solution based on the multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm. The proposed model can dynamically assign ROW for sidewalks, autonomous vehicles (AVs) driving lanes and on-street parking areas in real-time. Integrated with the SUMO traffic simulator, this model was evaluated using the road network of the South Kensington District against three cases of divergent traffic conditions: pedestrian flow rates, AVs traffic flow rates and parking demands. Results reveal that our model can achieve an average reduction of 3.87% and 6.26% in street space assigned for on-street parking and vehicular operations. Combined with space gained by limiting the number of driving lanes, the average proportion of sidewalks to total widths of streets can significantly increase by 10.13%.

Cooperation between agents in a multi-agent system (MAS) has become a hot topic in recent years, and many algorithms based on centralized training with decentralized execution (CTDE), such as VDN and QMIX, have been proposed. However, these methods disregard the information hidden in the individual action values. In this paper, we propose HyperGraph CoNvolution MIX (HGCN-MIX), a method that combines hypergraph convolution with value decomposition. By treating action values as signals, HGCN-MIX aims to explore the relationship between these signals via a self-learning hypergraph. Experimental results present that HGCN-MIX matches or surpasses state-of-the-art techniques in the StarCraft II multi-agent challenge (SMAC) benchmark on various situations, notably those with a number of agents.

The best virtual agents can avoid the need for some tickets to be submitted to help desk by offering up information that a user can use to self-service an issue or by resolving the issue directly by triggering an automated workflow such as a password reset. When a ticket does need to be submitted a virtual agent can ensure that tickets are fully-formed and “actionable”. Virtual agents can also relieve analysts from chasing down users for more information and handling repeatable mundane requests. It shouldn’t come as a surprise, therefore, that many of the most forward-looking IT organizations are already leveraging virtual agents to transform their service desks. Who Is Using Them? What is interesting, perhaps, is that these organizations are not limited to any one industry or type of business. Here are a few examples: City government—wanted to accelerate issue resolution and improve IT costs with a conversational approach to issue determination and resolution; deployed a virtual agent to drive self-service uptake and improve automation; and expect a 30% improvement in costs, along with enhancements to user satisfaction and service desk productivity. Global software company—hoped to streamline user interactions by completing and triaging tickets (or deflecting them altogether), and surveying users once issues have been resolved; installed a virtual agent to optimize the number of requests directed to its service desk catalog and automate as many workflows as possible; and forecast a 35% improvement in overall support costs. Managed service provider—decided to offer a key customer a solution which improved user satisfaction by reducing telephone support wait times; implemented a virtual agent in front of the customer’s interactive voice response system to divert users’ inquiries to a self-service knowledge base whenever possible; predict a 30% reduction in call volumes and a similar improvement in customer wait times. Multinational electronics manufacturer—resolved to increase the productivity of employees by enabling them to report issues on mobile devices removing the need for them to leave the manufacturing floor to access a computer terminal to do so; deploy a virtual agent as a first point of contact, enabling simple requests to be diverted to a knowledge database and issues resolved intuitively; anticipate at least a 30% improvement in support and service costs. State government—elected to improve the adoption of self-service resolution of issues by providing a more intuitive way for users to obtain assistance; installed a virtual agent as a conversational interface with simple issues routed to relevant sections of a help desk knowledge base for self-service resolution; expecting a 3x increase in the amount of issues resolved without analyst involvement. The ability of the best virtual agents to have an impact in such a wide variety of businesses and governments is, in part, due to them being agnostic to the ITSM environments into which they are deployed. They can, consequently, be deployed anywhere. Insight and Learnings Luma, the virtual agent that we’ve developed here at Serviceaide, and purpose-built for IT service management, already integrates seamlessly with leading ITSM solutions from  CA Technologies, Cherwell, Freshservice and ServiceNow, and we will be are adding other leading ITSM platforms. And, of course, Luma also connects to our own Intelligent Service Management solution. Organizations, across a variety of industry sectors, have deployed Luma, or are in the process of doing so. I’m excited about these successes and look forward to sharing more details about them with you in future blog articles – I’m especially eager to describe the interesting things we’ve learned during the development and onboarding process in each instance, as I’m sure that this will be very insightful to others about to embark on a virtual agent deployment. Thanks for reading. We hope that our blog articles can inform and start conversations. If this article piques your interest, but leaves you wanting more, let me know.

Hyperparameter optimization (HPO) is a necessary step to ensure the best possible performance of Machine Learning (ML) algorithms. Several methods have been developed to perform HPO; most of these are focused on optimizing one performance measure (usually an error-based measure), and the literature on such single-objective HPO problems is vast. Recently, though, algorithms have appeared which focus on optimizing multiple conflicting objectives simultaneously. This article presents a systematic survey of the literature published between 2014 and 2020 on multi-objective HPO algorithms, distinguishing between metaheuristic-based algorithms, metamodel-based algorithms, and approaches using a mixture of both. We also discuss the quality metrics used to compare multi-objective HPO procedures and present future research directions.

Market makers play a key role in financial markets by providing liquidity. They usually fill order books with buy and sell limit orders in order to provide traders alternative price levels to operate. This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective. In particular, we propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets. This research analyzes how RL market maker agents behaves in non-competitive (only one RL market maker learning at the same time) and competitive scenarios (multiple RL market markers learning at the same time), and how they adapt their strategies in a Sim2Real scope with interesting results. Furthermore, it covers the application of policy transfer between different experiments, describing the impact of competing environments on RL agents performance. RL and deep RL techniques are proven as profitable market maker approaches, leading to a better understanding of their behavior in stock markets.

We consider online resource allocation under a typical non-profit setting, where limited or even scarce resources are administered by a not-for-profit organization like a government. We focus on the internal-equity by assuming that arriving requesters are homogeneous in terms of their external factors like demands but heterogeneous for their internal attributes like demographics. Specifically, we associate each arriving requester with one or several groups based on their demographics (i.e., race, gender, and age), and we aim to design an equitable distributing strategy such that every group of requesters can receive a fair share of resources proportional to a preset target ratio. We present two LP-based sampling algorithms and investigate them both theoretically (in terms of competitive-ratio analysis) and experimentally based on real COVID-19 vaccination data maintained by the Minnesota Department of Health. Both theoretical and numerical results show that our LP-based sampling strategies can effectively promote equity, especially when the arrival population is disproportionately represented, as observed in the early stage of the COVID-19 vaccine rollout.

Automatically adapting game content to players opens new doors for game development. In this paper we propose an architecture using persona agents and experience metrics, which enables evolving procedurally generated levels tailored for particular player personas. Using our game, "Grave Rave", we demonstrate that this approach successfully adapts to four rule-based persona agents over three different experience metrics. Furthermore, the adaptation is shown to be specific in nature, meaning that the levels are persona-conscious, and not just general optimizations with regard to the selected metric.

We consider distributed online min-max resource allocation with a set of parallel agents and a parameter server. Our goal is to minimize the pointwise maximum over a set of time-varying convex and decreasing cost functions, without a priori information about these functions. We propose a novel online algorithm, termed Distributed Online resource Re-Allocation (DORA), where non-stragglers learn to relinquish resource and share resource with stragglers. A notable feature of DORA is that it does not require gradient calculation or projection operation, unlike most existing online optimization strategies. This allows it to substantially reduce the computation overhead in large-scale and distributed networks. We show that the dynamic regret of the proposed algorithm is upper bounded by $O\left(T^{\frac{3}{4}}(1+P_T)^{\frac{1}{4}}\right)$, where $T$ is the total number of rounds and $P_T$ is the path-length of the instantaneous minimizers. We further consider an application to the bandwidth allocation problem in distributed online machine learning. Our numerical study demonstrates the efficacy of the proposed solution and its performance advantage over gradient- and/or projection-based resource allocation algorithms in reducing wall-clock time.

In this research, we extend the universal reinforcement learning (URL) agent models of artificial general intelligence to quantum environments. The utility function of a classical exploratory stochastic Knowledge Seeking Agent, KL-KSA, is generalized to distance measures from quantum information theory on density matrices. Quantum process tomography (QPT) algorithms form the tractable subset of programs for modeling environmental dynamics. The optimal QPT policy is selected based on a mutable cost function based on algorithmic complexity as well as computational resource complexity. Instead of Turing machines, we estimate the cost metrics on a high-level language to allow realistic experimentation. The entire agent design is encapsulated in a self-replicating quine which mutates the cost function based on the predictive value of the optimal policy choosing scheme. Thus, multiple agents with pareto-optimal QPT policies evolve using genetic programming, mimicking the development of physical theories each with different resource trade-offs. This formal framework is termed Quantum Knowledge Seeking Agent (QKSA). Despite its importance, few quantum reinforcement learning models exist in contrast to the current thrust in quantum machine learning. QKSA is the first proposal for a framework that resembles the classical URL models. Similar to how AIXI-tl is a resource-bounded active version of Solomonoff universal induction, QKSA is a resource-bounded participatory observer framework to the recently proposed algorithmic information-based reconstruction of quantum mechanics. QKSA can be applied for simulating and studying aspects of quantum information theory. Specifically, we demonstrate that it can be used to accelerate quantum variational algorithms which include tomographic reconstruction as its integral subroutine.