Goto

Collaborating Authors

 resource utilization




FedCLF -- Towards Efficient Participant Selection for Federated Learning in Heterogeneous IoV Networks

Wijethilake, Kasun Eranda, Mahmood, Adnan, Sheng, Quan Z.

arXiv.org Artificial Intelligence

Federated Learning (FL) is a distributed machine learning technique that preserves data privacy by sharing only the trained parameters instead of the client data. This makes FL ideal for highly dynamic, heterogeneous, and time-critical applications, in particular, the Internet of Vehicles (IoV) networks. However, FL encounters considerable challenges in such networks owing to the high data and device heterogeneity. To address these challenges, we propose FedCLF, i.e., FL with Calibrated Loss and Feedback control, which introduces calibrated loss as a utility in the participant selection process and a feedback control mechanism to dynamically adjust the sampling frequency of the clients. The envisaged approach (a) enhances the overall model accuracy in case of highly heterogeneous data and (b) optimizes the resource utilization for resource constrained IoV networks, thereby leading to increased efficiency in the FL process. We evaluated FedCLF vis-à-vis baseline models, i.e., FedAvg, Newt, and Oort, using CIFAR-10 dataset with varying data heterogeneity. Our results depict that FedCLF significantly outperforms the baseline models by up to a 16% improvement in high data heterogeneity-related scenarios with improved efficiency via reduced sampling frequency.


PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture

Liu, Yi, Liu, Yang, Zheng, Leqian, Hong, Jue, Shi, Junjie, Yang, Qingyou, Wu, Ye, Wang, Cong

arXiv.org Artificial Intelligence

With the rapid advancement of the digital economy, data collaboration between organizations has become a well-established business model, driving the growth of various industries. However, privacy concerns make direct data sharing impractical. To address this, Two-Party Split Learning (a.k.a. Vertical Federated Learning (VFL)) has emerged as a promising solution for secure collaborative learning. Despite its advantages, this architecture still suffers from low computational resource utilization and training efficiency. Specifically, its synchronous dependency design increases training latency, while resource and data heterogeneity among participants further hinder efficient computation. To overcome these challenges, we propose PubSub-VFL, a novel VFL paradigm with a Publisher/Subscriber architecture optimized for two-party collaborative learning with high computational efficiency. PubSub-VFL leverages the decoupling capabilities of the Pub/Sub architecture and the data parallelism of the parameter server architecture to design a hierarchical asynchronous mechanism, reducing training latency and improving system efficiency. Additionally, to mitigate the training imbalance caused by resource and data heterogeneity, we formalize an optimization problem based on participants' system profiles, enabling the selection of optimal hyperparameters while preserving privacy. We conduct a theoretical analysis to demonstrate that PubSub-VFL achieves stable convergence and is compatible with security protocols such as differential privacy. Extensive case studies on five benchmark datasets further validate its effectiveness, showing that, compared to state-of-the-art baselines, PubSub-VFL not only accelerates training by $2 \sim 7\times$ without compromising accuracy, but also achieves a computational resource utilization rate of up to 91.07%.


Part II: ROLL Flash -- Accelerating RLVR and Agentic Training with Asynchrony

Lu, Han, Liu, Zichen, Xiong, Shaopan, He, Yancheng, Gao, Wei, Wu, Yanan, Wang, Weixun, Liu, Jiashun, Li, Yang, Zhao, Haizhou, Huang, Ju, Yang, Siran, Li, Xiaoyang, Luo, Yijia, Liu, Zihe, Pan, Ling, Yan, Junchi, Wang, Wei, Su, Wenbo, Wang, Jiamang, Qu, Lin, Zheng, Bo

arXiv.org Artificial Intelligence

Synchronous Reinforcement Learning (RL) post-training has emerged as a crucial step for enhancing Large Language Models (LLMs) with diverse capabilities. However, many systems designed to accelerate RL post-training still suffer from low resource utilization and limited scalability. We present ROLL Flash, a system that extends ROLL with native support for asynchronous RL post-training. ROLL Flash is built upon two core design principles: fine-grained parallelism and rollout-train decoupling. Guided by these principles, ROLL Flash provides flexible programming interfaces that enable a fully asynchronous training architecture and support efficient rollout mechanisms, including queue scheduling and environment-level asynchronous execution. Through comprehensive theoretical analysis and extensive experiments, we demonstrate that ROLL Flash significantly improves resource utilization and scalability over synchronous RL post-training. ROLL Flash achieves up to 2.24x speedup on RLVR tasks and 2.72x on agentic tasks, using the same GPU budget as synchronous baselines. Furthermore, we implement several popular off-policy algorithms and verify that asynchronous training can achieve performance on par with synchronous training.


Artificial Intelligence for Cost-Aware Resource Prediction in Big Data Pipelines

Goyal, Harshit

arXiv.org Artificial Intelligence

Efficient resource allocation is a key challenge in modern cloud computing. Over-provisioning leads to unnecessary costs, while under-provisioning risks performance degradation and SLA violations. This work presents an artificial intelligence approach to predict resource utilization in big data pipelines using Random Forest regression. We preprocess the Google Borg cluster traces to clean, transform, and extract relevant features (CPU, memory, usage distributions). The model achieves high predictive accuracy (R Square = 0.99, MAE = 0.0048, RMSE = 0.137), capturing non-linear relationships between workload characteristics and resource utilization. Error analysis reveals impressive performance on small-to-medium jobs, with higher variance in rare large-scale jobs. These results demonstrate the potential of AI-driven prediction for cost-aware autoscaling in cloud environments, reducing unnecessary provisioning while safeguarding service quality.


Learning Fairness in Multi-Agent Systems

Neural Information Processing Systems

Fairness is essential for human society, contributing to stability and productivity. Similarly, fairness is also the key for many multi-agent systems.


10493aa88605cad5ab4752b04a63d172-AuthorFeedback.pdf

Neural Information Processing Systems

We gratefully appreciate the efforts made by all the reviewers. Hughes et al. [2018] extend the inequity aversion model and define a shaped reward These works aim to improve cooperation but cannot guarantee fairness. We compare against Hughes et al. [2018], More details will be included in the final version. To verify the effectiveness of the hierarchy, we use the hierarchy with other baselines in job scheduling. That demonstrates the effect of the hierarchy. The intuition of the fair-efficient reward is to maximize the resource utilization while punish the agent's utility deviation The main hyperparameters are contained in the Appendix, we will make a further supplement in the final version.


Predicting Case Suffixes With Activity Start and End Times: A Sweep-Line Based Approach

Ali, Muhammad Awais, Dumas, Marlon, Milani, Fredrik

arXiv.org Artificial Intelligence

Predictive process monitoring techniques support the operational decision-making by predicting future states of ongoing cases of a business process. A subset of these techniques predict the remaining sequence of activities of an ongoing case (case suffix prediction). Existing approaches for case suffix prediction generate sequences of activities with a single timestamp (e.g. the end timestamp). This output is insufficient for resource capacity planning, where we need to reason about the periods of time when resources will be busy performing work. This paper introduces a technique for predicting case suffixes consisting of activities with start and end timestamps. In other words, the proposed technique predicts both the waiting time and the processing time of each activity. Since the waiting time of an activity in a case depends on how busy resources are in other cases, the technique adopts a sweep-line approach, wherein the suffixes of all ongoing cases in the process are predicted in lockstep, rather than predictions being made for each case in isolation. An evaluation on real-life and synthetic datasets compares the accuracy of different instantiations of this approach, demonstrating the advantages of a multi-model approach to case suffix prediction.


Prediction of mortality and resource utilization in critical care: a deep learning approach using multimodal electronic health records with natural language processing techniques

Ruan, Yucheng, Lan, Xiang, Tan, Daniel J., Abdullah, Hairil Rizal, Feng, Mengling

arXiv.org Artificial Intelligence

Background Predicting mortality and resource utilization from electronic health records (EHRs) is challenging yet crucial for optimizing patient outcomes and managing costs in intensive care unit (ICU). Existing approaches predominantly focus on structured EHRs, often ignoring the valuable clinical insights in free-text notes. Additionally, the potential of textual information within structured data is not fully leveraged. This study aimed to introduce and assess a deep learning framework using natural language processing techniques that integrates multimodal EHRs to predict mortality and resource utilization in critical care settings. Methods Utilizing two real-world EHR datasets, we developed and evaluated our model on three clinical tasks with leading existing methods. We also performed an ablation study on three key components in our framework: medical prompts, free-texts, and pre-trained sentence encoder. Furthermore, we assessed the model's robustness against the corruption in structured EHRs. Results Our experiments on two real-world datasets across three clinical tasks showed that our proposed model improved performance metrics by 1.6\%/0.8\% on BACC/AUROC for mortality prediction, 0.5%/2.2% on RMSE/MAE for LOS prediction, 10.9%/11.0% on RMSE/MAE for surgical duration estimation compared to the best existing methods. It consistently demonstrated superior performance compared to other baselines across three tasks at different corruption rates. Conclusions The proposed framework is an effective and accurate deep learning approach for predicting mortality and resource utilization in critical care. The study also highlights the success of using prompt learning with a transformer encoder in analyzing multimodal EHRs. Importantly, the model showed strong resilience to data corruption within structured data, especially at high corruption levels.