BenchMake: Turn any scientific data set into a reproducible benchmark

Benchmark data sets are curated collections that enable consistent, reproducible, and objective evaluation of algorithms and models [1, 2]. They are essential for comparing algorithm performance fairly, particularly in machine learning (ML) and artificial intelligence (AI), where the suitability of algorithms can vary widely based on data structure, dimensionality, and distribution [3, 4]. For instance, algorithms that perform exceptionally on structured, tabular data may not generalise well to unstructured image or textual data [5]. Established benchmarks such as ImageNet [2], CIFAR data sets [6], and OpenML benchmarks for structured data [7] have driven innovation by providing clear metrics for progress, fostering reproducibility and trust within the research community [8]. However, in computational sciences, standardised benchmarks remain rare and challenging to establish due to the intrinsic complexity, heterogeneity, and domain specificity of scientific data [9]. Scientific data sets can be represented in a variety of ways (tables, image, text, graphs, signals), often requiring extensive pre-processing, specialised evaluation metrics, and are subject to measurement noise, natural variability, and data imbalance [10].