Federated learning (FL) enables multiple data owners (a.k.a. FL clients) to collaboratively train machine learning models without disclosing sensitive private data. Existing FL research mostly focuses on the monopoly scenario in which a single FL server selects a subset of FL clients to update their local models in each round of training. In practice, there can be multiple FL servers simultaneously trying to select clients from the same pool. In this paper, we propose a first-of-its-kind Fairness-aware Federated Job Scheduling (FairFedJS) approach to bridge this gap. Based on Lyapunov optimization, it ensures fair allocation of high-demand FL client datasets to FL jobs in need of them, by jointly considering the current demand and the job payment bids, in order to prevent prolonged waiting. Extensive experiments comparing FairFedJS against four state-of-the-art approaches on two datasets demonstrate its significant advantages. It outperforms the best baseline by 31.9% and 1.0% on average in terms of scheduling fairness and convergence time, respectively, while achieving comparable test accuracy.

In this critical survey, we analyze typical claims on the relationship between explainable AI (XAI) and fairness to disentangle the multidimensional relationship between these two concepts. Based on a systematic literature review and a subsequent qualitative content analysis, we identify seven archetypal claims from 175 papers on the alleged fairness benefits of XAI. We present crucial caveats with respect to these claims and provide an entry point for future discussions around the potentials and limitations of XAI for specific fairness desiderata. Importantly, we notice that claims are often (i) vague and simplistic, (ii) lacking normative grounding, or (iii) poorly aligned with the actual capabilities of XAI. We encourage to conceive XAI not as an ethical panacea but as one of many tools to approach the multidimensional, sociotechnical challenge of algorithmic fairness. Moreover, when making a claim about XAI and fairness, we emphasize the need to be more specific about what kind of XAI method is used and which fairness desideratum it refers to, how exactly it enables fairness, and who is the stakeholder that benefits from XAI.

Effective patient monitoring is vital for timely interventions and improved healthcare outcomes. Traditional monitoring systems often struggle to handle complex, dynamic environments with fluctuating vital signs, leading to delays in identifying critical conditions. To address this challenge, we propose a novel AI-driven patient monitoring framework using multi-agent deep reinforcement learning (DRL). Our approach deploys multiple learning agents, each dedicated to monitoring a specific physiological feature, such as heart rate, respiration, and temperature. These agents interact with a generic healthcare monitoring environment, learn the patients' behaviour patterns, and make informed decisions to alert the corresponding Medical Emergency Teams (METs) based on the level of emergency estimated. In this study, we evaluate the performance of the proposed multi-agent DRL framework using real-world physiological and motion data from two datasets: PPG-DaLiA and WESAD. We compare the results with several baseline models, including Q-Learning, PPO, Actor-Critic, Double DQN, and DDPG, as well as monitoring frameworks like WISEML and CA-MAQL. Our experiments demonstrate that the proposed DRL approach outperforms all other baseline models, achieving more accurate monitoring of patient's vital signs. Furthermore, we conduct hyperparameter optimization to fine-tune the learning process of each agent. By optimizing hyperparameters, we enhance the learning rate and discount factor, thereby improving the agents' overall performance in monitoring patient health status.

This paper investigates the collision-free control problem for multi-agent systems. For such multi-agent systems, it is the typical situation where conventional methods using either the usual centralized model predictive control (MPC), or even the distributed counterpart, would suffer from substantial difficulty in balancing optimality and computational efficiency. Additionally, the non-convex characteristics that invariably arise in such collision-free control and optimization problems render it difficult to effectively derive a reliable solution (and also to thoroughly analyze the associated convergence properties). To overcome these challenging issues, this work establishes a suitably novel parallel computation framework through an innovative mathematical problem formulation; and then with this framework and formulation, a parallel algorithm based on alternating direction method of multipliers (ADMM) is presented to solve the sub-problems arising from the resulting parallel structure. Furthermore, an efficient and intuitive initialization procedure is developed to accelerate the optimization process, and the optimum is thus determined with significantly improved computational efficiency. As supported by rigorous proofs, the convergence of the proposed ADMM iterations for this non-convex optimization problem is analyzed and discussed in detail. Finally, a simulation with a group of unmanned aerial vehicles (UAVs) serves as an illustrative example here to demonstrate the effectiveness and efficiency of the proposed approach. Also, the simulation results verify significant improvements in accuracy and computational efficiency compared to other baselines, including primal quadratic mixed integer programming (PQ-MIP), non-convex quadratic mixed integer programming (NC-MIP), and non-convex quadratically constrained quadratic programming (NC-QCQP).

In reinforcement learning, reward shaping is an efficient way to guide the learning process of an agent, as the reward can indicate the optimal policy of the task. The potential-based reward shaping framework was proposed to guarantee policy invariance after reward shaping, where a potential function is used to calculate the shaping reward. In former work, we proposed a novel adaptive potential function (APF) method to learn the potential function concurrently with training the agent based on information collected by the agent during the training process, and examined the APF method in discrete action space scenarios. This paper investigates the feasibility of using APF in solving continuous-reaching tasks in a real-world robotic scenario with continuous action space. We combine the Deep Deterministic Policy Gradient (DDPG) algorithm and our proposed method to form a new algorithm called APF-DDPG. To compare APF-DDPG with DDPG, we designed a task where the agent learns to control Baxter's right arm to reach a goal position. The experimental results show that the APF-DDPG algorithm outperforms the DDPG algorithm on both learning speed and robustness.

In some settings, although unbiased data from the target domain remains human demonstration videos can be easily recorded in a controllable a challenge due to costly data collection processes manner in the target environment, the distinct embodiment and stringent safety requirements. Consequently, from the target robot agents hinders their direct use researchers often resort to data from easily accessible in policy learning (Yu et al., 2018). Such intricate environment source domains, such as simulation and laboratory and embodiment discrepancies, also referred to as domain environments, for cost-effective data acquisition gaps, negatively impact policies trained on source domain and rapid model iteration. Nevertheless, the data and inevitably lead to their deployment failures in environments and embodiments of these source domains the target domains. The data bottlenecks in real-world tasks can be quite different from their target domain and the wide existence of domain gaps naturally stimulated counterparts, underscoring the need for effective cross-domain policy transfer studies, aiming to fully exploit cross-domain policy transfer approaches. In existing off-domain data to learn transferable policies.

Reinforcement learning (RL) with sparse and deceptive rewards is challenging because non-zero rewards are rarely obtained. Hence, the gradient calculated by the agent can be stochastic and without valid information. Recent studies that utilize memory buffers of previous experiences can lead to a more efficient learning process. However, existing methods often require these experiences to be successful and may overly exploit them, which can cause the agent to adopt suboptimal behaviors. This paper develops an approach that uses diverse past trajectories for faster and more efficient online RL, even if these trajectories are suboptimal or not highly rewarded. The proposed algorithm combines a policy improvement step with an additional exploration step using offline demonstration data. The main contribution of this paper is that by regarding diverse past trajectories as guidance, instead of imitating them, our method directs its policy to follow and expand past trajectories while still being able to learn without rewards and approach optimality. Furthermore, a novel diversity measurement is introduced to maintain the team's diversity and regulate exploration. The proposed algorithm is evaluated on discrete and continuous control tasks with sparse and deceptive rewards. Compared with the existing RL methods, the experimental results indicate that our proposed algorithm is significantly better than the baseline methods regarding diverse exploration and avoiding local optima.

We explore the AI2050 "hard problems" that block the promise of AI and cause AI risks: (1) developing general capabilities of the systems; (2) assuring the performance of AI systems and their training processes; (3) aligning system goals with human goals; (4) enabling great applications of AI in real life; (5) addressing economic disruptions; (6) ensuring the participation of all; (7) at the same time ensuring socially responsible deployment; (8) addressing any geopolitical disruptions that AI causes; (9) promoting sound governance of the technology; and (10) managing the philosophical disruptions for humans living in the age of AI. For each problem, we outline the area, identify significant recent work, and suggest ways forward. [Note: this paper reviews literature through January 2023.]

As the potential for societal integration of multi-agent robotic systems increases [1], the need to manage the collective behaviors of such systems also increases [2, 3, 4]. There has been significant research effort directed towards the examination of how humans can assist in controlling such collective behaviors, such as in human-swarm interactions [5, 6, 7]. Agent-agent interactions in a swarm of small unmanned aerial systems (sUAS) lead to the emergence of collective behaviors that enable effective coverage and exploration across large spatial extents. However, the same inherent collective behaviors can occasionally limit the ability of the sUAS swarm to focus on specific objects of interest during coverage or exploration missions [8]. In these scenarios, the human operator or supervisor should have the opportunity to fractionally revoke or limit emergent swarm behaviors, and guide the swarm to achieve mission objectives. For most applications, including in industry-and defense-related contexts, such human-swarm interaction (HSI) will likely require intuitive and predictable mechanisms of control to quickly translate the input of the human (such as a gesture) to an influence or effect on the sUAS swarm. The goal of our work is to create an intuitive interface for a human supervisor to influence or guide an sUAS swarm without excessive incursions on decentralized control afforded by these systems, while attempting to create more predictable behaviors. This is a potentially valuable approach that can enable the fully utilization of swarm capabilities, while also retaining an ongoing macroscopic-level of swarm control in scenarios where focus on specific regions of interest is required (e.g., search and rescue, surveillance operations) [9]. The influence mechanism has been implemented and tested using 16 drones in a photo-realistic virtual reality (VR) environment (as shown in Figure 1).

Multirotor UAVs are used for a wide spectrum of civilian and public domain applications. Navigation controllers endowed with different attributes and onboard sensor suites enable multirotor autonomous or semi-autonomous, safe flight, operation, and functionality under nominal and detrimental conditions and external disturbances, even when flying in uncertain and dynamically changing environments. During the last decade, given the faster-than-exponential increase of available computational power, different learning-based algorithms have been derived, implemented, and tested to navigate and control, among other systems, multirotor UAVs. Learning algorithms have been, and are used to derive data-driven based models, to identify parameters, to track objects, to develop navigation controllers, and to learn the environment in which multirotors operate. Learning algorithms combined with model-based control techniques have been proven beneficial when applied to multirotors. This survey summarizes published research since 2015, dividing algorithms, techniques, and methodologies into offline and online learning categories, and then, further classifying them into machine learning, deep learning, and reinforcement learning sub-categories. An integral part and focus of this survey are on online learning algorithms as applied to multirotors with the aim to register the type of learning techniques that are either hard or almost hard real-time implementable, as well as to understand what information is learned, why, and how, and how fast. The outcome of the survey offers a clear understanding of the recent state-of-the-art and of the type and kind of learning-based algorithms that may be implemented, tested, and executed in real-time.