Reinforcement Learning-driven Data-intensive Workflow Scheduling for Volunteer Edge-Cloud

In recent times, Volunteer Edge-Cloud (VEC) has gained traction as a cost-effective, community computing paradigm to support data-intensive scientific workflows. However, due to the highly distributed and heterogeneous nature of VEC resources, centralized workflow task scheduling remains a challenge. In this paper, we propose a Reinforcement Learning (RL)-driven dataintensive scientific workflow scheduling approach that takes into consideration: i) workflow requirements, ii) VEC resources' preference on workflows, and iii) diverse VEC resource policies, to ensure robust resource allocation. We formulate the long-term average performance optimization problem as a Markov Decision Process, which is solved using an event-based Asynchronous Advantage Actor-Critic based RL approach. Our extensive simulations and testbed implementations demonstrate our approach's benefits over popular baseline strategies in terms of workflow requirement satisfaction, VEC preference satisfaction, and available VEC resource utilization. Data-intensive scientific workflows in areas characterized by considerable on-demand resource needs and stringent security requirements (e.g., bioinformatics, high-energy physics, and healthcare), have traditionally been hosted by cloud environments, thanks to the availability of resources, advanced security protocols, and performance assurances through Service Level Agreements (SLAs) [1] offered by such environments. To address this, in recent times, "volunteer edge-cloud" (VEC) computing has emerged as an alternative [2], [3], harnessing distributed computing to provide cost-effective resources [4] for on-demand processing. Figure 1 illustrates an exemplary VEC environment that leverages the collective computational resources of VEC nodes (i.e., VNs) to process data-intensive workflows; thereby shifting the processing from centralized cloud infrastructures to the edge, where resources are more affordable and abundant, albeit diverse and geographically distributed. These VNs can range from small devices (e.g., IoTs) to large systems (e.g., servers) that are owned and operated by individuals, laboratories, or organizations who willingly contribute them for collaborative computing.