Bayesian adaptive experimental design is a form of active learning, which chooses samples to maximize the information they give about uncertain parameters. Prior work has shown that other forms of active learning can suffer from active learning bias, where unrepresentative sampling leads to inconsistent parameter estimates. We show that active learning bias can also afflict Bayesian adaptive experimental design, depending on model misspecification. We analyze the case of estimating a linear model, and show that worse misspecification implies more severe active learning bias. At the same time, model classes incorporating more "noise" -- i.e., specifying higher inherent variance in observations -- suffer less from active learning bias. Finally, we demonstrate empirically that insights from the linear model can predict the presence and degree of active learning bias in nonlinear contexts, namely in a (simulated) preference learning experiment. Statistical theory often assumes learners' access to large amounts of representative training data, drawn from the distribution which is the target of inference or prediction. Nonetheless, such access is not feasible for many applications. Training data may be scarce (e.g., learning to identify a rare medical condition; Henry, Hager, Pronovost, and Saria (2015)), difficult or expensive to obtain (e.g., requiring human coders for text; Chen, Lasko, Mei, Denny, and Xu (2015)), or time-consuming to collect (e.g., obtaining user preferences online; Cavagnaro, Gonzalez, Myung, and Pitt (2013); Golovin, Krause, and Ray (2010)). One response is to abandon random sampling for adaptive sampling methods, choosing data points in sequence to be as informative as possible.

Farquhar et al. [2021] show that correcting for active learning bias with underparameterised models leads to improved downstream performance. For overparameterised models such as NNs, however, correction leads either to decreased or unchanged performance. They suggest that this is due to an "overfitting bias" which offsets the active learning bias. We show that depth uncertainty networks operate in a low overfitting regime, much like underparameterised models. They should therefore see an increase in performance with bias correction. Surprisingly, they do not. We propose that this negative result, as well as the results Farquhar et al. [2021], can be explained via the lens of the bias-variance decomposition of generalisation error.

Active learning is a powerful tool when labelling data is expensive, but it introduces a bias because the training data no longer follows the population distribution. We formalize this bias and investigate the situations in which it can be harmful and sometimes even helpful. We further introduce novel corrective weights to remove bias when doing so is beneficial. Through this, our work not only provides a useful mechanism that can improve the active learning approach, but also an explanation of the empirical successes of various existing approaches which ignore this bias. In particular, we show that this bias can be actively helpful when training overparameterized models -- like neural networks -- with relatively little data.