Neural network training consists largely of matrix multiplications that generally account for the vast majority of the computational cost for standard architectures such as transformers.

C.1 2DSyntheticBenchmark For both benchmarks, we sample 500 observationsxi=(x1i,x2i)from each of the twoin-domain classes (orange and blue), and consider a deep architecture ResFFN-12-128, which contains 12 residual feedforward layers with 128 hidden units and dropout rate 0.01.

Efficient methods that reliably quantify a deep neural network (DNN)'s predictive uncertainty are important for industrial-scale, real-world applications, which include examples such as object recognition in autonomous driving [22], ad click prediction in online advertising [76], and intent understanding inaconversational system [84].

Segmentation of curvilinear structures such as vasculature and road networks is challenging due to relatively weak signals and complex geometry/topology.