Machine Learning-Driven Adaptive OpenMP For Portable Performance on Heterogeneous Systems

The end of Dennard scaling law -- which stipulated a continuous increase in processor clock frequency by transistor miniaturization -- in conjunction with the continuation of Moore's law -- which expects the number of CMOS transistors within a microchip to double every two years -- shifted the technology trend towards parallel architectures. In the early 2000's parallel computer system architectures focused on multi-core CPU architectures. Later the introduction of the GPGPU paradigms pivoted technology trends to heterogeneous systems composed of both multi-core CPUs and GPUs. This heterogeneity unveiled the challenge of software performance portability. Software performance portability seeks to achieve equivalent performance regardless of the underlying hardware architecture using a single application implementation. Programming models, such as OmpSs [9], OpenMP, Kokkos [10], and RAJA [15], provide abstractions to hide the vendor-specific interfaces required to develop applications on all these heterogeneous parallel architectures and offer unified interfaces to express parallelism. Although these programming models provide a single and convenient layer to implement portable code, the performance of the same application can vary when executed on different architectures and systems. Thus, these programming models efficiently express portable code, but the application performance-portability is unspecified for application executions on different heterogeneous systems. For example, HPC programmers have found that a single version of source code, with an associated static definition of exarXiv:2303.08873v1