HPC Storage Service Autotuning Using Variational-Autoencoder-Guided Asynchronous Bayesian Optimization

They range from Empirical performance tuning, also known as autotuning, is multiuser, high-speed storage systems such as burst buffers [2], a hot topic in software optimization nowadays, and a promising [3], [4], to transient, application-specific services providing approach for HPC storage service tuning. In this approach, processing capabilities such as in situ analysis [5], [6], [7]. the user exposes the tunable parameters and defines the range These systems aim to improve I/O and storage performance of values that each parameter can take; a search method by moving away from file-based interfaces and from the is then used to explore the parameter space by executing POSIX semantics, instead providing specific interfaces and optimizations different parameter configurations on the target platform. The that can be tailored to individual applications. An challenge for HPC storage services autotuning stems from example of such a distributed storage service is HEPnOS [8], the complexity of the workflow and the search space. First, an in-memory object store for high-energy physics (HEP) several tunable parameters can be interdependent, requiring an applications developed by Argonne National Laboratory and execution of the complete workflow on the target platform for FermiLab.