Suppose an n d design matrix in a linear regression problem is given, but the response for each point is hidden unless explicitly requested. The goal is to sample only a small number k n of the responses, and then produce a weight vector whose sum of squares loss over all points is at most 1+ times the minimum. When k is very small (e.g., k = d), jointly sampling diverse subsets of points is crucial. One such method called volume sampling has a unique and desirable property that the weight vector it produces is an unbiased estimate of the optimum. It is therefore natural to ask if this method offers the optimal unbiased estimate in terms of the number of responses k needed to achieve a 1+ loss approximation. Surprisingly we show that volume sampling can have poor behavior when we require a very accurate approximation - indeed worse than some i.i.d.

Neural Information Processing Systems http://nips.cc/