Appendix Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation A Code

In Figure 2, we examine the probability of acquiring a '7' as a function of the number of acquired We see that XWED initially focuses on 7s but then diversifies. The XWED behavior is preferable: we are initially unsure about the loss of these points, but once the loss is well characterized for the 7s we should explore other areas as well. B.2 Constant π Fails for Distribution Shift. Figure B.1 (a) shows that, for LURE suffered high variance in Figure 3. In Figure B.1 (b), we observe that ASE continues to Figure B.2 demonstrates that ASEs continue to outperform all other baselines for the task of This result highlights the importance of the adaptive nature of both ASE-and LUREbased active testing. Figure B.2: V ariant of the experiments of 7.3 where we estimate the accuracy of the main model. We here investigate a variation of the experiments in 7.3: reducing the size of the training set to Despite this, Figure B.3 demonstrates that ASEs continue to outperform all baselines.