Supplementary Material AEvaluation on CIFARBenchmarks

Setup We additionally evaluate GradNorm on a common benchmark with CIFAR-10 and CIFAR100 [22] as ID datasets, which is routinely used in literature [13, 27, 14, 29, 26]. We use the standard split with 50,000 training images and 10,000 test images. The learning rate is initially 0.1, and decays by a factor of 10 at epochs 50, 75 and 90 respectively. Results We summarize the results in Table 6, where GradNormremains competitive. In particular, GradNorm reduces the average FPR95 by 8.77% on CIFAR-10 compared to the best baseline.