The agent pulls an arm and observes the corresponding return provided by the environment.

Ournew bound consistently beats state-of-the-art bounds both on a toy example and on UCI datasets (with large enoughn).

We exploit the richstructures ofthereformulated problem anddevelopastochastic primal-dual algorithm, SVRPDA-I, to solve the problem efficiently.

Weprove its pointwise universality, i.e., it achieves a normalized log loss performance asymptotically as good as the true conditional entropy of the labels given the features.

In this section we discuss how to adapt existing linear bandit results (mostly in the regret setting) to sample complexity results.

The unit ℓ2-spherein d-dimensions that is centered at the origin is denoted bySd 1. Additionally, given a pair of symmetric matricesA,B Rd, we write A B if and only if x (A B)x 0, x Rd. More linear algebra facts appear in AppendixE. Let V P be a subset of distributions indexed by the points in the hypercubeEd = { 1,1}d. For a number of facts from probability and statistics (both related and unrelated to exponential families),wereferthereadertoAppendixF.