Merely pursuing performance may adversely affect the safety, while a conservative policy for safe exploration will degrade the performance. How to balance the safety and performance in learning-based control problems is an interesting yet challenging issue. This paper aims to enhance system performance with safety guarantee in solving the reinforcement learning (RL)-based optimal control problems of nonlinear systems subject to high-relative-degree state constraints and unknown time-varying disturbance/actuator faults. First, to combine control barrier functions (CBFs) with RL, a new type of CBFs, termed high-order reciprocal control barrier function (HO-RCBF) is proposed to deal with high-relative-degree constraints during the learning process. Then, the concept of gradient similarity is proposed to quantify the relationship between the gradient of safety and the gradient of performance. Finally, gradient manipulation and adaptive mechanisms are introduced in the safe RL framework to enhance the performance with a safety guarantee. Two simulation examples illustrate that the proposed safe RL framework can address high-relative-degree constraint, enhance safety robustness and improve system performance.

Abstract--Industrial Internet of Things (IIoT) applications demand efficient task offloading to handle heavy data loads with minimal latency. Mobile Edge Computing (MEC) brings computation closer to devices to reduce latency and server load, optimal performance requires advanced optimization techniques. We propose a novel solution combining Adaptive Particle Swarm Optimization (APSO) with Reinforcement Learning, specifically Soft Actor Critic (SAC), to enhance task offloading decisions in MEC environments. This hybrid approach leverages swarm intelligence and predictive models to adapt to dynamic variables such as human interactions and environmental changes. Our method improves resource management and service quality, achieving optimal task offloading and resource distribution in IIoT edge computing.

As Large Language Models (LLMs) advance in their capabilities, researchers have increasingly employed them for social simulation. In this paper, we investigate whether interactions among LLM agents resemble those of humans. Specifically, we focus on the pronoun usage difference between leaders and non-leaders, examining whether the simulation would lead to human-like pronoun usage patterns during the LLMs' interactions. Our evaluation reveals the significant discrepancies between LLM-based simulations and human pronoun usage, with prompt-based or specialized agents failing to demonstrate human-like pronoun usage patterns. In addition, we reveal that even if LLMs understand the human pronoun usage patterns, they fail to demonstrate them in the actual interaction process. Our study highlights the limitations of social simulations based on LLM agents, urging caution in using such social simulation in practitioners' decision-making process.

Retrieval-augmented generation (RAG) is extensively utilized to incorporate external, current knowledge into large language models, thereby minimizing hallucinations. A standard RAG pipeline may comprise several components, such as query rewriting, document retrieval, document filtering, and answer generation. However, these components are typically optimized separately through supervised fine-tuning, which can lead to misalignments between the objectives of individual modules and the overarching aim of generating accurate answers in question-answering (QA) tasks. Although recent efforts have explored reinforcement learning (RL) to optimize specific RAG components, these approaches often focus on overly simplistic pipelines with only two components or do not adequately address the complex interdependencies and collaborative interactions among the modules. To overcome these challenges, we propose treating the RAG pipeline as a multi-agent cooperative task, with each component regarded as an RL agent. Specifically, we present MMOA-RAG, a Multi-Module joint Optimization Algorithm for RAG, which employs multi-agent reinforcement learning to harmonize all agents' goals towards a unified reward, such as the F1 score of the final answer. Experiments conducted on various QA datasets demonstrate that MMOA-RAG improves the overall pipeline performance and outperforms existing baselines. Furthermore, comprehensive ablation studies validate the contributions of individual components and the adaptability of MMOA-RAG across different RAG components and datasets. The code of MMOA-RAG is on https://github.com/chenyiqun/MMOA-RAG.

Summary and Contributions: The authors provide a regret-minimisation approach to computing an analogue to correlated equilibria in extensive form games called extensive-form correlated equilibria (EFCE). It was previously unknown whether EFCE can be achieved via uncoupled no-regret dynamics as in typical correlated equilibria in simultaneous games, and the authors provide a way of doing so by introducing an appropriate notion of regret in the extensive form setting (that lines up with the notion of approximation in approximate EFCE), and demonstrating how achieving low regret in this setting suffices to have an approximate EFCE for joint strategy profiles that arise from empirical frequencies of play. As mentioned before, the relevant notion of equilibrium in this setting are extensive-form correlated equilibria (EFCE). Such an equilibrium is a joint distribution over the space of all possible plans of all agents. As is typical in an extensive-form setting, a plan is simply a mapping from information sets to their relevant action profiles that dictates what an agent does at any given situation of play. In the simultaneous setting, a mixed strategy profile is a correlated equilibrium when no agent wishes to deviate from the joint strategy profile, conditional on their realised strategy profile and prior knowledge of the joint distribution of play.

We study federated contextual linear bandits, where M agents cooperate with each other to solve a global contextual linear bandit problem with the help of a central server. We consider the asynchronous setting, where all agents work independently and the communication between one agent and the server will not trigger other agents' communication. We propose a simple algorithm named FedLinUCB based on the principle of optimism. We prove that the regret of FedLinUCB is bounded by \widetilde{\mathcal{O}}(d\sqrt{\sum_{m 1} M T_m}) and the communication complexity is \widetilde{O}(dM 2), where d is the dimension of the contextual vector and T_m is the total number of interactions with the environment by agent m . To the best of our knowledge, this is the first provably efficient algorithm that allows fully asynchronous communication for federated linear bandits, while achieving the same regret guarantee as in the single-agent setting.

Weaknesses: I believe the results proposed in this paper are related to existing work. The techniques used are close to existing methods - at the very least a detailed comparison is in order. The paper fails to acknowledge lots of literature on representing coalitional games in a restricted manner. In fact, many techniques have been proposed for concisely representing coalitional games, and approximately solving them. This issue is covered in depth in (e.g): Chalkiadakis, Georgios, Edith Elkind, and Michael Wooldridge.

Summary and Contributions: Summary of contributions i) They set out to deploy probabilistic methods to determine the probability of dangerous events and determine the safety of a given, where dangerous events are simulated in a custom-built simulator, that combines exploration, exploitation, and optimization techniques to find failure modes and estimate the rate of occurrence. Summary They combine an adapted version of HMC, that they call warped HMC which, through sequential updates, utilizes normalizing flows and bridge sampling to extract samples corresponding to rare-events in a variety of different scenarios, generated via stochastic simulation. This paper shares some similar themes with NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport, but they also combine a series of other techniques. I had read this two-weeks ago and contributed to the discussions, so I apologise for the delay in the update. Just a few points and I believe the AC/ other reviewers have provided you with more feedback.

The paper proposes a method for empirical verification of safety system by developing an iterative method for sampling rare and potentially out of bounds system states. The reviewers agree that the strengths are in the novel and important area that is generally little worked in. The method is well founded, rigorously analyzed, and evaluated thoroughly. The methodology presented in the paper is general and applicable to other applications other than the safety, and can generate a lot of follow-up work. In the final version the authors should: - Address R1's concerns about ReLU non-reversibility and show the validity of the bridge-sampling hybrid algorithm empirically or analytically.

Robots have shown enormous potential to alleviate repetitive, and dangerous tasks from human workers, such as assembly, infrastructure inspection, material handling and heavy rigging [4-6]. Integrating the artificial intelligence (AI) agent with a physical robotic system could further improve the precision, reliability, and consistency of operations with competent training [7, 8]. While AI-enabled robots excel in performing repetitive and predefined tasks, dexterous and complex tasks still pose a significant difficulty such as welding and pipe insertion [9, 10]. Training a robot to perform these dexterous tasks demands delicate manipulation and adaptive force control, which induces diversity and several potential actions leading to a substantial increase in the complexity of the learning process and resulting in slow convergence or lack of convergence [11] To tackle the challenges of learning in high-dimensional action spaces, Imitation Learning (IL) based methods are applied to leverage demonstrations from human experts or proficient use of human demonstrations as a form of instruction and reduce the size of action spaces that need to be explored [12-14]. Generative Adversarial Imitation Learning (GAIL)[15] could further address some key limitations of traditional IL by mitigating distributional shifts, thus enabling better exploration and performance in unseen states and generalizing better to new tasks [15].