Agents
Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents
Agashe, Saaket, Wong, Kyle, Tu, Vincent, Yang, Jiachen, Li, Ang, Wang, Xin Eric
Computer use agents automate digital tasks by directly interacting with graphical user interfaces (GUIs) on computers and mobile devices, offering significant potential to enhance human productivity by completing an open-ended space of user queries. However, current agents face significant challenges: imprecise grounding of GUI elements, difficulties with long-horizon task planning, and performance bottlenecks from relying on single generalist models for diverse cognitive tasks. To this end, we introduce Agent S2, a novel compositional framework that delegates cognitive responsibilities across various generalist and specialist models. We propose a novel Mixture-of-Grounding technique to achieve precise GUI localization and introduce Proactive Hierarchical Planning, dynamically refining action plans at multiple temporal scales in response to evolving observations. Evaluations demonstrate that Agent S2 establishes new state-of-the-art (SOTA) performance on three prominent computer use benchmarks. Specifically, Agent S2 achieves 18.9% and 32.7% relative improvements over leading baseline agents such as Claude Computer Use and UI-TARS on the OSWorld 15-step and 50-step evaluation. Moreover, Agent S2 generalizes effectively to other operating systems and applications, surpassing previous best methods by 52.8% on WindowsAgentArena and by 16.52% on AndroidWorld relatively. Code available at https://github.com/simular-ai/Agent-S.
First Field-Trial Demonstration of L4 Autonomous Optical Network for Distributed AI Training Communication: An LLM-Powered Multi-AI-Agent Solution
Zhang, Yihao, Qiu, Qizhi, Liu, Xiaomin, Fu, Dianxuan, Liu, Xingyu, Fei, Leyan, Cheng, Yuming, Yi, Lilin, Hu, Weisheng, Zhuge, Qunbi
Abstract: We demonstrate the first cross - domain cross - layer level - 4 autonomous optical network via a multi - AI - agent system. Field trials show ~ 9 8 % task completion rate across the distributed AI training lifecycle -- 3.2 higher than single agents using state - of - the - art LLMs. Since collaborative resource utilization across distributed facilities is essential for training workloads, t his evolution introduces significant complexity in network management, as controller s must operate across multiple domains, spanning from intra - and inter - datacenter s to long - haul wide area networks . Moreover, distributed training impose s stringent reliability requirements as it should restart from the checkpoint if a failure happens [ 2 ] . T herefore, in terms of distributed training communications, resilient operations and rapid fault recovery are essential .
Remember, but also, Forget: Bridging Myopic and Perfect Recall Fairness with Past-Discounting
Dynamic resource allocation in multi-agent settings often requires balancing efficiency with fairness over time--a challenge inadequately addressed by conventional, myopic fairness measures. Motivated by behavioral insights that human judgments of fairness evolve with temporal distance, we introduce a novel framework for temporal fairness that incorporates past-discounting mechanisms. By applying a tunable discount factor to historical utilities, our approach interpolates between instantaneous and perfect-recall fairness, thereby capturing both immediate outcomes and long-term equity considerations. Beyond aligning more closely with human perceptions of fairness, this past-discounting method ensures that the augmented state space remains bounded, significantly improving computational tractability in sequential decision-making settings. We detail the formulation of discounted-recall fairness in both additive and averaged utility contexts, illustrate its benefits through practical examples, and discuss its implications for designing balanced, scalable resource allocation strategies.
HERA: Hybrid Edge-cloud Resource Allocation for Cost-Efficient AI Agents
Liu, Shiyi, Shen, Haiying, Che, Shuai, Ghandi, Mahdi, Li, Mingqin
In the realm of AI, large language models (LLMs) like GPT-4, central to the operation of AI agents, predominantly operate in the cloud, incurring high operational costs. With local-based small language models (SLMs) becoming more accurate, the necessity of cloud-exclusive processing is being reconsidered. An AI agent's response to a user's request comprises a series of subtasks or iterations. Existing approaches only allocate a single request between SLM and LLM to ensure their outputs are similar, but adopting this approach in the AI agent scenario for assigning each subtask is not effective since SLM will output a different subsequent subtask, which affects the accuracy of the final output. In this paper, we first conduct experimental analysis to understand the features of AI agent operations. Leveraging our findings, we propose the Adaptive Iteration-level Model Selector (AIMS), a lightweight scheduler to automatically partition AI agent's subtasks between local-based SLM and cloud-based LLM. AIMS considers the varying subtask features and strategically decides the location for each subtask in order to use SLM as much as possible while attaining the accuracy level. Our experimental results demonstrate that AIMS increases accuracy by up to 9.1% and SLM usage by up to 10.8% compared to HybridLLM. It offloads 45.67% of subtasks to a local SLM while attaining similar accuracy on average compared with the cloud-only LLM approach.
Simulation of Autonomous Industrial Vehicle Fleet Using Fuzzy Agents: Application to Task Allocation and Battery Charge Management
Grosset, Juliette, Fougรจres, Alain-Jรฉrรดme, Oukacha, Ouzna, Djoko-Kouam, Moรฏse, Bonnin, Jean-Marie
Abstract: The research introduces a multi - agent simulation that uses fuzzy inference to investigate the work distribution and battery charging control of mobile baggage conveyor robots in an airport in a comprehensive manner. Thanks to a distributed system, this simulation approach provides high adaptability, adjusting to changes in conveyor agent availability, battery capacity, awareness of the activities of the conveyor fleet, and knowledge of the context of infrastructure resource availability. Dynamic factors, such as workload variations and communication between the conveyor agents and infrastructure are con sidered as heuristics, hig hlighting the importance of flexible and collaborative approaches in autonomous systems. The results highlight the effectiveness of adaptive fuzzy multi - agent models to optimize dynamic task allocation, adapt to the variation of baggage arrival flows, impr ove the overall operational efficiency of conveyor agents, and reduce their energy consumption. Keywords: autonomous industrial vehicle, agent - based si mulation, fuzzy agent, dynamic task allocation, battery charge management, Airport 4.0 1. INTRODUCTION The implementation of fleets of Autonomous Industrial Vehicles (AIV) in the context of Airport 4.0 presents a number of challenges, all of which are connected to the true degree of autonomy of these vehicles: employee acceptance, vehicle localization, traf fic flow, failure detection, collision avoidance, and vehicle perception in dynamic environments. The different limitations and specifications developed by producers and potential consumers of these AIVs might be taken into consideration thanks to simulati on.
AgentNet: Decentralized Evolutionary Coordination for LLM-based Multi-Agent Systems
Yang, Yingxuan, Chai, Huacan, Shao, Shuai, Song, Yuanyi, Qi, Siyuan, Rui, Renting, Zhang, Weinan
The rapid advancement of Large Language Models (LLMs) has catalyzed the development of multi-agent systems, where multiple LLM-based agents collaborate to solve complex tasks. However, existing systems predominantly rely on centralized coordination, which introduces scalability bottlenecks, limits adaptability, and creates single points of failure. Additionally, concerns over privacy and proprietary knowledge sharing hinder cross-organizational collaboration, leading to siloed expertise. To address these challenges, we propose AgentNet, a decentralized, Retrieval-Augmented Generation (RAG)-based framework that enables LLM-based agents to autonomously evolve their capabilities and collaborate efficiently in a Directed Acyclic Graph (DAG)-structured network. Unlike traditional multi-agent systems that depend on static role assignments or centralized control, AgentNet allows agents to specialize dynamically, adjust their connectivity, and route tasks without relying on predefined workflows. AgentNet's core design is built upon several key innovations: (1) Fully Decentralized Paradigm: Removing the central orchestrator, allowing agents to coordinate and specialize autonomously, fostering fault tolerance and emergent collective intelligence. (2) Dynamically Evolving Graph Topology: Real-time adaptation of agent connections based on task demands, ensuring scalability and resilience.(3) Adaptive Learning for Expertise Refinement: A retrieval-based memory system that enables agents to continuously update and refine their specialized skills. By eliminating centralized control, AgentNet enhances fault tolerance, promotes scalable specialization, and enables privacy-preserving collaboration across organizations. Through decentralized coordination and minimal data exchange, agents can leverage diverse knowledge sources while safeguarding sensitive information.
Off-Policy Evaluation for Sequential Persuasion Process with Unobserved Confounding
S., Nishanth Venkatesh, Bang, Heeseung, Malikopoulos, Andreas A.
-- In this paper, we expand the Bayesian persuasion framework to account for unobserved confounding variables in sender-receiver interactions. While traditional models typically assume that belief updates follow Bayesian principles, real-world scenarios often involve hidden variables that impact the receiver's belief formation and decision-making. Crucially, the receiver's belief update is affected by an unobserved confounding variable. By reformulating this scenario as a Partially Observable Markov Decision Process (POMDP), we capture the sender's incomplete information regarding both the dynamics of the receiver's beliefs and the unobserved confounder . We prove that finding an optimal observation-based policy in this POMDP is equivalent to solving for an optimal signaling strategy in the original persuasion framework. Furthermore, we demonstrate how this reformulation facilitates the application of proximal learning for off-policy evaluation (OPE) in the persuasion process. This advancement enables the sender to evaluate alternative signaling strategies using only observational data from a behavioral policy, thus eliminating the necessity for costly new experiments. Strategic information sharing plays a critical role in economic interactions, policy design, and multi-agent systems [1]-[3]. Bayesian persuasion was first introduced by Ka-menica and Gentzkow [4] as a powerful framework for analyzing how a sender can strategically reveal information to influence a receiver's decisions. In the standard setting, a sender commits to an information disclosure policy before observing the state of the world, and the receiver, after observing the sender's message, forms posterior beliefs and takes an action that affects both the sender's and the receiver's utilities. Despite its theoretical elegance, Bayesian persuasion rests on assumptions that may not hold in practical settings. First, the framework presupposes that the sender possesses complete information about the receiver, including their observation process and all features that influence their decision-making (including utility functions).
GraphMaster: Automated Graph Synthesis via LLM Agents in Data-Limited Environments
Du, Enjun, Li, Xunkai, Jin, Tian, Zhang, Zhihan, Li, Rong-Hua, Wang, Guoren
The era of foundation models has revolutionized AI research, yet Graph Foundation Models (GFMs) remain constrained by the scarcity of large-scale graph corpora. Traditional graph data synthesis techniques primarily focus on simplistic structural operations, lacking the capacity to generate semantically rich nodes with meaningful textual attributes: a critical limitation for real-world applications. While large language models (LLMs) demonstrate exceptional text generation capabilities, their direct application to graph synthesis is impeded by context window limitations, hallucination phenomena, and structural consistency challenges. To address these issues, we introduce GraphMaster, the first multi-agent framework specifically designed for graph data synthesis in data-limited environments. GraphMaster orchestrates four specialized LLM agents (Manager, Perception, Enhancement, and Evaluation) that collaboratively optimize the synthesis process through iterative refinement, ensuring both semantic coherence and structural integrity. To rigorously evaluate our approach, we create new data-limited "Sub" variants of six standard graph benchmarks, specifically designed to test synthesis capabilities under realistic constraints. Additionally, we develop a novel interpretability assessment framework that combines human evaluation with a principled Grassmannian manifold-based analysis, providing both qualitative and quantitative measures of semantic coherence. Experimental results demonstrate that GraphMaster significantly outperforms traditional synthesis methods across multiple datasets, establishing a strong foundation for advancing GFMs in data-scarce environments.
Egocentric Conformal Prediction for Safe and Efficient Navigation in Dynamic Cluttered Environments
Shin, Jaeuk, Lee, Jungjin, Yang, Insoon
Since safe control of ego-vehicles depends on accurately predicting the future states of surrounding dynamic agents, numerous motion forecasting models [1, 2] have been developed to forecast an agent's future motions from historical data. Nevertheless, these predictions remain inherently prone to error, primarily because they lack information about hidden contexts or intents--such as agents' goals, velocity preferences, or even social relationships among human agents. To address these limitations, conformal prediction (CP) [3, 4] has been employed to reliably assess the models' predictive capabilities. The method offers a principled yet straightforward procedure for calibrating the models. At test time, the calibration results can be used to construct a confidence set that contains the true future states of the environment, assuming that the test and calibration data are exchangeable (i.e., their joint distribution is symmetric). Consequently, CP has been successfully applied to a variety of problems, including reinforcement learning [5, 6], linear This work was supported in part by the Information and Communications Technology Planning and Evaluation (IITP) grants funded by MSIT No. 2022-0-00124, No. 2022-0-00480 and No. RS-2021-II211343, Artificial Intelligence Graduate School Program (Seoul National University). The authors are with the Department of Electrical and Computer Engineering, ASRI, Seoul National University, Seoul 08826, South Korea,{sju5379, jungbbal, insoonyang }@snu.ac.kr arXiv:2504.00447v1
Value Iteration for Learning Concurrently Executable Robotic Control Tasks
Tahmid, Sheikh A., Notomista, Gennaro
Many modern robotic systems such as multi-robot systems and manipulators exhibit redundancy, a property owing to which they are capable of executing multiple tasks. This work proposes a novel method, based on the Reinforcement Learning (RL) paradigm, to train redundant robots to be able to execute multiple tasks concurrently. Our approach differs from typical multi-objective RL methods insofar as the learned tasks can be combined and executed in possibly time-varying prioritized stacks. We do so by first defining a notion of task independence between learned value functions. We then use our definition of task independence to propose a cost functional that encourages a policy, based on an approximated value function, to accomplish its control objective while minimally interfering with the execution of higher priority tasks. This allows us to train a set of control policies that can be executed simultaneously. We also introduce a version of fitted value iteration to learn to approximate our proposed cost functional efficiently. We demonstrate our approach on several scenarios and robotic systems.