Upland helps global businesses accelerate digital transformation with a powerful cloud software library that provides choice, flexibility, and value. Our growing library of products delivers the plug-in processes, reporting, and job specific workflows that major cloud platforms and homegrown systems don't provide. We focus on specific business challenges and support every corner of the organization, operating at scale and delivering quick time to value for our 1,800 enterprise customers. We are experts in knowledge management, trusted by major organizations globally to deliver Connected Knowledge to internal staff and customers. Through years of development and partnerships with leading software vendors, getting the right answers to the right people at the right time, we have created out-of-the-box integrations that can connect multiple repositories alongside the native knowledge base, and display them in the users CRM/ITSM tools or dedicated portals.

In this extended abstract we discuss the opportunities and challenges of studying intrinsically-motivated agents for exploration in textual environments. We argue that there is important synergy between text environments and autonomous agents. We identify key properties of text worlds that make them suitable for exploration by autonmous agents, namely, depth, breadth, progress niches and the ease of use of language goals; we identify drivers of exploration for such agents that are implementable in text worlds. We discuss the opportunities of using autonomous agents to make progress on text environment benchmarks. Finally we list some specific challenges that need to be overcome in this area.

We present CompoSuite, an open-source simulated robotic manipulation benchmark for compositional multi-task reinforcement learning (RL). Each CompoSuite task requires a particular robot arm to manipulate one individual object to achieve a task objective while avoiding an obstacle. This compositional definition of the tasks endows CompoSuite with two remarkable properties. First, varying the robot/object/objective/obstacle elements leads to hundreds of RL tasks, each of which requires a meaningfully different behavior. Second, RL approaches can be evaluated specifically for their ability to learn the compositional structure of the tasks. This latter capability to functionally decompose problems would enable intelligent agents to identify and exploit commonalities between learning tasks to handle large varieties of highly diverse problems. We benchmark existing single-task, multi-task, and compositional learning algorithms on various training settings, and assess their capability to compositionally generalize to unseen tasks. Our evaluation exposes the shortcomings of existing RL approaches with respect to compositionality and opens new avenues for investigation.

Classical medium access control (MAC) protocols are interpretable, yet their task-agnostic control signaling messages (CMs) are ill-suited for emerging mission-critical applications. By contrast, neural network (NN) based protocol models (NPMs) learn to generate task-specific CMs, but their rationale and impact lack interpretability. To fill this void, in this article we propose, for the first time, a semantic protocol model (SPM) constructed by transforming an NPM into an interpretable symbolic graph written in the probabilistic logic programming language (ProbLog). This transformation is viable by extracting and merging common CMs and their connections while treating the NPM as a CM generator. By extensive simulations, we corroborate that the SPM tightly approximates its original NPM while occupying only 0.02% memory. By leveraging its interpretability and memory-efficiency, we demonstrate several SPM-enabled applications such as SPM reconfiguration for collision-avoidance, as well as comparing different SPMs via semantic entropy calculation and storing multiple SPMs to cope with non-stationary environments. Traditionally, cellular medium access control (MAC) protocols have been designed primarily for general purposes. Ko is with Inha University, Incheon, Korea (e-mail: swko@inha.ac.kr). This work has been submitted to the IEEE for possible publication. While handshaking rules and scheduling policies can partly be manipulated (e.g., grant-free access prioritization [2]), their control signaling messages (CMs) remain unchanged even when tasks and other environmental characteristics vary over time.

Regret has been established as a foundational concept in online learning, and likewise has important applications in the analysis of learning dynamics in games. Regret quantifies the difference between a learner’s performance against a baseline in hindsight. It is well known that regret-minimizing algorithms converge to certain classes of equilibria in games; however, traditional forms of regret used in game theory predominantly consider baselines that permit deviations to deterministic actions or strategies. In this paper, we revisit our understanding of regret from the perspective of deviations over partitions of the full mixed strategy space (i.e., probability distributions over pure strategies), under the lens of the previously-established Φ-regret framework, which provides a continuum of stronger regret measures. Importantly, Φ-regret enables learning agents to consider deviations from and to mixed strategies, generalizing several existing notions of regret such as external, internal, and swap regret, and thus broadening the insights gained from regret-based analysis of learning algorithms. We prove here that the well-studied evolutionary learning algorithm of replicator dynamics (RD) seamlessly minimizes the strongest possible form of Φ-regret in generic 2 × 2 games, without any modification of the underlying algorithm itself. We subsequently conduct experiments validating our theoretical results in a suite of 144 2 × 2 games wherein RD exhibits a diverse set of behaviors. We conclude by providing empirical evidence of Φ-regret minimization by RD in some larger games, hinting at further opportunity for Φ-regret based study of such algorithms from both a theoretical and empirical perspective.

The particle swarm optimization (PSO) algorithm is a population-based search algorithm based on the simulation of the social behavior of birds within a flock. The initial intent of the particle swarm concept was to graphically simulate the graceful and unpredictable choreography of a bird flock, to discover patterns that govern the ability of birds to fly synchronously, and to suddenly change direction by regrouping in an optimal formation. From this initial objective, the concept evolved into a simple and efficient optimization algorithm. So, just like the Genetic Algorithm, PSO is inspired by nature. In PSO, individuals, also referred to as particles, are "flown" through hyperdimensional search space. Changes to the position of particles within the search space are based on the social-psychological tendency of individuals to emulate the success of other individuals.

Planning problems are, in general, PSPACE-complete; large problems, especially multi-agent problems with required coordination, can be intractable or impractical to solve. Factored planning and multi-agent planning both address this by separating multi-agent problems into tractable sub-problems, but there are limitations in the expressivity of existing planners and in the ability to handle tightly coupled multi-agent problems. This paper presents EGOPLAN, a framework which factors a multi-agent problem into related sub-problems which are solved by iteratively calling on a single agent planner. EGOPLAN is evaluated on a multi-robot test domain with durative actions, required coordination, and temporal constraints, comparing the performance of a temporal planner, OPTIC-CPLEX, with and without EGOPLAN. Our results show that for our test domain, using EGOPLAN allows OPTIC-CPLEX to solve problems that are twice as complex as it can solve without EGOPLAN, and to solve complex problems significantly faster.

In this paper, we propose a novel benchmark called the StarCraft Multi-Agent Challenges+, where agents learn to perform multi-stage tasks and to use environmental factors without precise reward functions. The previous challenges (SMAC) recognized as a standard benchmark of Multi-Agent Reinforcement Learning are mainly concerned with ensuring that all agents cooperatively eliminate approaching adversaries only through fine manipulation with obvious reward functions. This challenge, on the other hand, is interested in the exploration capability of MARL algorithms to efficiently learn implicit multi-stage tasks and environmental factors as well as micro-control. This study covers both offensive and defensive scenarios. In the offensive scenarios, agents must learn to first find opponents and then eliminate them. The defensive scenarios require agents to use topographic features. For example, agents need to position themselves behind protective structures to make it harder for enemies to attack. We investigate MARL algorithms under SMAC+ and observe that recent approaches work well in similar settings to the previous challenges, but misbehave in offensive scenarios. Additionally, we observe that an enhanced exploration approach has a positive effect on performance but is not able to completely solve all scenarios. This study proposes new directions for future research.

We study federated contextual linear bandits, where $M$ agents cooperate with each other to solve a global contextual linear bandit problem with the help of a central server. We consider the asynchronous setting, where all agents work independently and the communication between one agent and the server will not trigger other agents' communication. We propose a simple algorithm named \texttt{FedLinUCB} based on the principle of optimism. We prove that the regret of \texttt{FedLinUCB} is bounded by $\tilde{O}(d\sqrt{\sum_{m=1}^M T_m})$ and the communication complexity is $\tilde{O}(dM^2)$, where $d$ is the dimension of the contextual vector and $T_m$ is the total number of interactions with the environment by $m$-th agent. To the best of our knowledge, this is the first provably efficient algorithm that allows fully asynchronous communication for federated contextual linear bandits, while achieving the same regret guarantee as in the single-agent setting.

Due to the COVID-19 pandemic, the global supply chain is disrupted at an unprecedented scale under uncertain and unknown trends of labor shortage, high material prices, and changing travel or trade regulations. To stay competitive, enterprises desire agile and dynamic response strategies to quickly react to disruptions and recover supply-chain functions. Although both centralized and multi-agent approaches have been studied, their implementation requires prior knowledge of disruptions and agent-rule-based reasoning. In this paper, we introduce a model-based multi-agent framework that enables agent coordination and dynamic agent decision-making to respond to supply chain disruptions in an agile and effective manner. Through a small-scale simulated case study, we showcase the feasibility of the proposed approach under several disruption scenarios that affect a supply chain network differently, and analyze performance trade-offs between the proposed distributed and centralized methods.