A Design space

Compute resources We trained the configurations on a large SLURM-based cluster with approximately 300,000 CPU-cores available in parallel. Data splits We split our performance dataset into a training, validation and test split in an approximately 70-15-15 ratio. In step 1, we treated every single configuration's data points across multiple epochs as time-series data, where each epoch is a single time step, thereby grouping together Adding bounds Since XGBoost is an unbounded regression model i.e. its codomain is This allows for a comprehensive analysis of optimizer's performance Dataset Average predicted runtime [CPU-d] CIFAR-10 2.0 Colorectal-Histology 0.2 Fashion-MNIST 2.2 This does not take into account carbon emissions for optimizing and training the surrogate benchmarks based on the data and indirect emissions such as creating the compute hardware and maintenance of the compute cluster. We noted that our surrogate models' performance on the Colorectal-Histology task was much worse In the first experiment, 20 configurations were randomly chosen from the set of configurations belonging to the highest fidelity group (N=5, W=16, R=1.0) that had already been evaluated on CIFAR-10 and Colorectal-Histology for our performance dataset and re-evaluated for 200 epochs using 2 different, randomly sampled sets of seeds for initialization. We present the results of this analysis in Table 11.