There is an increasing need for effective active learning algorithms that are compatible with deep neural networks.

Work performed while at Stanford University.

We illustrate these bounds in the logistic regression and neural network settings. In contrast, we establish settings where learning one representation for all tasks (i.e. using a "frozen representation" objective) fails. Notably, any such algorithm cannot outperform directly learning the target task with no other information, in the worst case.