However, for many real-world problems it is not enough to just minimize regret on a particular problem instance. For example, suppose we have run an online education experiment using a bandit algorithm where we test different types of teaching strategies.

We study the relationship between the notions of differentially private learning and online learning in games. Several recent works have shown that differentially private learning implies online learning, but an open problem of Neel, Roth, and Wu [27] asks whether this implication is efficient. Specifically, does an efficient differentially private learner imply an efficient online learner? In this paper we resolve this open question in the context of pure differential privacy. We derive an efficient black-box reduction from differentially private learning to online learning from expert advice.

In this paper, we consider online learning in generalized linear contextual bandits where rewards are not immediately observed.

We consider a sequential learning problem with Gaussian payoffs and side observations: after selecting an action i, the learner receives information about the payoff of every action j in the form of Gaussian observations whose mean is the same as the mean payoff, but the variance depends on the pair (i,j) (and may be infinite). The setup allows a more refined information transfer from one action to another than previous partial monitoring setups, including the recently introduced graph-structured feedback case. For the first time in the literature, we provide non-asymptotic problem-dependent lower bounds on the regret of any algorithm, which recover existing asymptotic problem-dependent lower bounds and finite-time minimax lower bounds available in the literature. We also provide algorithms that achieve the problem-dependent lower bound (up to some universal constant factor) or the minimax lower bounds (up to logarithmic factors).

Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

Next, we consider two algorithms for the problem of prediction with expert advice over trees. The following lemma (whose proof is deferred to Appendix D) formally states this fact. Suppose that Algorithm 5 is run using predictions and updates provided by Algorithm 6. Suppose that Algorithm 5 is run using predictions and updates provided by AdaNormalHedge. We start by proving a master regret bound that can be specialized to various settings of interest. Combining terms completes the proof.

Neural Information Processing Systems http://nips.cc/