Globus Automation Services: Research process automation across the space-time continuum
Chard, Ryan, Pruyne, Jim, McKee, Kurt, Bryan, Josh, Raumann, Brigitte, Ananthakrishnan, Rachana, Chard, Kyle, Foster, Ian
–arXiv.org Artificial Intelligence
Research process automation -- the reliable, efficient, and reproducible execution of linked sets of actions on scientific instruments, computers, data stores, and other resources -- has emerged as an essential element of modern science. We report here on new services within the Globus research data management platform that enable the specification of diverse research processes as reusable sets of actions, \emph{flows}, and the execution of such flows in heterogeneous research environments. To support flows with broad spatial extent (e.g., from scientific instrument to remote data center) and temporal extent (from seconds to weeks), these Globus automation services feature: 1) cloud hosting for reliable execution of even long-lived flows despite sporadic failures; 2) a simple specification and extensible asynchronous action provider API, for defining and executing a wide variety of actions and flows involving heterogeneous resources; 3) an event-driven execution model for automating execution of flows in response to arbitrary events; and 4) a rich security model enabling authorization delegation mechanisms for secure execution of long-running actions across distributed resources. These services permit researchers to outsource and automate the management of a broad range of research tasks to a reliable, scalable, and secure cloud platform. We present use cases for Globus automation services, describe their design and implementation, present microbenchmark studies, and review experiences applying the services in a range of applications.
arXiv.org Artificial Intelligence
Dec-6-2022
- Country:
- North America > United States (0.93)
- Genre:
- Research Report (0.70)
- Workflow (0.48)
- Industry:
- Energy (0.93)
- Health & Medicine (1.00)
- Information Technology
- Security & Privacy (1.00)
- Services (1.00)
- Technology:
- Information Technology
- Artificial Intelligence
- Machine Learning > Neural Networks (0.46)
- Representation & Reasoning (0.67)
- Cloud Computing (1.00)
- Communications > Web (0.93)
- Data Science > Data Mining (0.67)
- Information Management (1.00)
- Software > Programming Languages (0.67)
- Software Engineering (1.00)
- Artificial Intelligence
- Information Technology