Agents
FinA: Fairness of Adverse Effects in Decision-Making of Human-Cyber-Physical-System
Ensuring fairness in decision-making systems within Human-Cyber-Physical-Systems (HCPS) is a pressing concern, particularly when diverse individuals, each with varying behaviors and expectations, coexist within the same application space, influenced by a shared set of control actions in the system. The long-term adverse effects of these actions further pose the challenge, as historical experiences and interactions shape individual perceptions of fairness. This paper addresses the challenge of fairness from an equity perspective of adverse effects, taking into account the dynamic nature of human behavior and evolving preferences while recognizing the lasting impact of adverse effects. We formally introduce the concept of Fairness-in-Adverse-Effects (FinA) within the HCPS context. We put forth a comprehensive set of five formulations for FinA, encompassing both the instantaneous and long-term aspects of adverse effects. To empirically validate the effectiveness of our FinA approach, we conducted an evaluation within the domain of smart homes, a pertinent HCPS application. The outcomes of our evaluation demonstrate that the adoption of FinA significantly enhances the overall perception of fairness among individuals, yielding an average improvement of 66.7% when compared to the state-of-the-art method.
Enabling In-Situ Resources Utilisation by leveraging collaborative robotics and astronaut-robot interaction
Romero-Azpitarte, Silvia, Luna, Cristina, Guerra, Alba, Alonso, Mercedes, Manrique, Pablo Romeo, Seoane, Marina L., Olayo, Daniel, Moreno, Almudena, Castellanos, Pablo, Gandรญa, Fernando, Visentin, Gianfranco
Space exploration and establishing human presence on other planets demand advanced technology and effective collaboration between robots and astronauts. Efficient space resource utilization is also vital for extraterrestrial settlements. The Collaborative In-Situ Resources Utilisation (CISRU) project has developed a software suite comprising five key modules. The first module manages multi-agent autonomy, facilitating communication between agents and mission control. The second focuses on environment perception, employing AI algorithms for tasks like environment segmentation and object pose estimation. The third module ensures safe navigation, covering obstacle avoidance, social navigation with astronauts, and cooperation among robots. The fourth module addresses manipulation functions, including multi-tool capabilities and tool-changer design for diverse tasks in In-Situ Resources Utilization (ISRU) scenarios. Finally, the fifth module controls cooperative behaviour, incorporating astronaut commands, Mixed Reality interfaces, map fusion, task supervision, and error control. The suite was tested using an astronaut-rover interaction dataset in a planetary environment and GMV SPoT analogue environments. Results demonstrate the advantages of E4 autonomy and AI in space systems, benefiting astronaut-robot collaboration. This paper details CISRU's development, field test preparation, and analysis, highlighting its potential to revolutionize planetary exploration through AI-powered technology.
CISRU: a robotics software suite to enable complex rover-rover and astronaut-rover interaction
Romero-Azpitarte, Silvia, Guerra, Alba, Alonso, Mercedes, Seoane, Marina L., Olayo, Daniel, Moreno, Almudena, Castellanos, Pablo, Luna, Cristina, Visentin, Gianfranco
This level of autonomy, in Space exploration, particularly the long-term habitation of conjunction with collaboration between astronauts and robots, planetary surfaces, requires significant technological advances, is pivotal for the successful construction of structures and the with a strong focus on collaboration between robots and astronauts accomplishment of mission-specific tasks. This paper presents where the modularity and autonomy of space robots will the development of the CISRU suite, the preparation of field stand out, allowing them to perform different tasks [2], [3].
Imitation Learning based Alternative Multi-Agent Proximal Policy Optimization for Well-Formed Swarm-Oriented Pursuit Avoidance
Li, Sizhao, Xiang, Yuming, Li, Rongpeng, Zhao, Zhifeng, Zhang, Honggang
Multi-Robot System (MRS) has garnered widespread research interest and fostered tremendous interesting applications, especially in cooperative control fields. Yet little light has been shed on the compound ability of formation, monitoring and defence in decentralized large-scale MRS for pursuit avoidance, which puts stringent requirements on the capability of coordination and adaptability. In this paper, we put forward a decentralized Imitation learning based Alternative Multi-Agent Proximal Policy Optimization (IA-MAPPO) algorithm to provide a flexible and communication-economic solution to execute the pursuit avoidance task in well-formed swarm. In particular, a policy-distillation based MAPPO executor is firstly devised to capably accomplish and swiftly switch between multiple formations in a centralized manner. Furthermore, we utilize imitation learning to decentralize the formation controller, so as to reduce the communication overheads and enhance the scalability. Afterwards, alternative training is leveraged to compensate the performance loss incurred by decentralization. The simulation results validate the effectiveness of IA-MAPPO and extensive ablation experiments further show the performance comparable to a centralized solution with significant decrease in communication overheads.
Secured Fiscal Credit Model: Multi-Agent Systems And Decentralized Autonomous Organisations For Tax Credit's Tracking
De Gasperis, Giovanni, Facchini, Sante Dino, Letteri, Ivan
Tax incentives and fiscal bonuses have had a significant impact on the Italian economy over the past decade. In particular, the "Superbonus 110" tax relief in 2020, offering a generous 110% deduction for expenses related to energy efficiency improvements and seismic risk reduction in buildings, has played a pivotal role. However, the surge in construction activities has also brought about an unfortunate increase in fraudulent activities. To address this challenge, our research introduces a practical system for monitoring and managing the entire process of the Superbonus 110 tax credit, from its initiation to redemption. This system leverages artificial intelligence and blockchain technology to streamline tax credit management and incorporates controllers based on a Decentralised Autonomous Organisation architecture, bolstered by a Multi-agent System. The outcome of our work is a system capable of establishing a tokenomics framework that caters to the needs and functionalities of both investors and operators. Moreover, it features a robust control system to prevent inadvertent errors like double spending, overspending, and deceitful practices such as false claims of completed work. The collaborative approach between the Decentralised Autonomous Organisation and the Multi-agent System enhances trust and security levels among participants in a competitive environment where potential fraudsters might attempt to exploit the system. It also enables comprehensive tracking and monitoring of the entire Superbonus process. In the realm of engineering, our project represents an innovative fusion of blockchain technology and Multi-agent Systems, advancing the application of artificial intelligence. This integration guarantees the validation, recording, and execution of transactions with a remarkable level of trust and transparency.
A Multi-Agent Reinforcement Learning Framework for Evaluating the U.S. Ending the HIV Epidemic Plan
Sharma, Dinesh, Shah, Ankit, Gopalappa, Chaitra
Human immunodeficiency virus (HIV) is a major public health concern in the United States, with about 1.2 million people living with HIV and 35,000 newly infected each year. There are considerable geographical disparities in HIV burden and care access across the U.S. The 2019 Ending the HIV Epidemic (EHE) initiative aims to reduce new infections by 90% by 2030, by improving coverage of diagnoses, treatment, and prevention interventions and prioritizing jurisdictions with high HIV prevalence. Identifying optimal scale-up of intervention combinations will help inform resource allocation. Existing HIV decision analytic models either evaluate specific cities or the overall national population, thus overlooking jurisdictional interactions or differences. In this paper, we propose a multi-agent reinforcement learning (MARL) model, that enables jurisdiction-specific decision analyses but in an environment with cross-jurisdictional epidemiological interactions. In experimental analyses, conducted on jurisdictions within California and Florida, optimal policies from MARL were significantly different than those generated from single-agent RL, highlighting the influence of jurisdictional variations and interactions. By using comprehensive modeling of HIV and formulations of state space, action space, and reward functions, this work helps demonstrate the strengths and applicability of MARL for informing public health policies, and provides a framework for expanding to the national-level to inform the EHE.
Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark
Ji, Jiaming, Zhang, Borong, Zhou, Jiayi, Pan, Xuehai, Huang, Weidong, Sun, Ruiyang, Geng, Yiran, Zhong, Yifan, Dai, Juntao, Yang, Yaodong
Artificial intelligence (AI) systems possess significant potential to drive societal progress. However, their deployment often faces obstacles due to substantial safety concerns. Safe reinforcement learning (SafeRL) emerges as a solution to optimize policies while simultaneously adhering to multiple constraints, thereby addressing the challenge of integrating reinforcement learning in safety-critical scenarios. In this paper, we present an environment suite called Safety-Gymnasium, which encompasses safety-critical tasks in both single and multi-agent scenarios, accepting vector and vision-only input. Additionally, we offer a library of algorithms named Safe Policy Optimization (SafePO), comprising 16 state-of-the-art SafeRL algorithms. This comprehensive library can serve as a validation tool for the research community. By introducing this benchmark, we aim to facilitate the evaluation and comparison of safety performance, thus fostering the development of reinforcement learning for safer, more reliable, and responsible real-world applications. The website of this project can be accessed at https://sites.google.com/view/safety-gymnasium.
SHAPE: A Framework for Evaluating the Ethicality of Influence
Bezou-Vrakatseli, Elfia, Brรผckner, Benedikt, Thorburn, Luke
Agents often exert influence when interacting with humans and non-human agents. However, the ethical status of such influence is often unclear. In this paper, we present the SHAPE framework, which lists reasons why influence may be unethical. We draw on literature from descriptive and moral philosophy and connect it to machine learning to help guide ethical considerations when developing algorithms with potential influence. Lastly, we explore mechanisms for governing algorithmic systems that influence people, inspired by mechanisms used in journalism, human subject research, and advertising.
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
Hong, Sirui, Zhuge, Mingchen, Chen, Jonathan, Zheng, Xiawu, Cheng, Yuheng, Zhang, Ceyao, Wang, Jinlin, Wang, Zili, Yau, Steven Ka Shing, Lin, Zijuan, Zhou, Liyang, Ran, Chenyu, Xiao, Lingfeng, Wu, Chenglin, Schmidhuber, Jรผrgen
Remarkable progress has been made on automated problem solving through societies of agents based on large language models (LLMs). Existing LLM-based multi-agent systems can already solve simple dialogue tasks. Solutions to more complex tasks, however, are complicated through logic inconsistencies due to cascading hallucinations caused by naively chaining LLMs. Here we introduce MetaGPT, an innovative meta-programming framework incorporating efficient human workflows into LLM-based multi-agent collaborations. MetaGPT encodes Standardized Operating Procedures (SOPs) into prompt sequences for more streamlined workflows, thus allowing agents with human-like domain expertise to verify intermediate results and reduce errors. MetaGPT utilizes an assembly line paradigm to assign diverse roles to various agents, efficiently breaking down complex tasks into subtasks involving many agents working together. On collaborative software engineering benchmarks, MetaGPT generates more coherent solutions than previous chat-based multi-agent systems. Our project can be found at https://github.com/geekan/MetaGPT
MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library
Hu, Siyi, Zhong, Yifan, Gao, Minquan, Wang, Weixun, Dong, Hao, Liang, Xiaodan, Li, Zhihui, Chang, Xiaojun, Yang, Yaodong
A significant challenge facing researchers in the area of multi-agent reinforcement learning (MARL) pertains to the identification of a library that can offer fast and compatible development for multi-agent tasks and algorithm combinations, while obviating the need to consider compatibility issues. In this paper, we present MARLlib, a library designed to address the aforementioned challenge by leveraging three key mechanisms: 1) a standardized multi-agent environment wrapper, 2) an agent-level algorithm implementation, and 3) a flexible policy mapping strategy. By utilizing these mechanisms, MARLlib can effectively disentangle the intertwined nature of the multi-agent task and the learning process of the algorithm, with the ability to automatically alter the training strategy based on the current task's attributes.