Optimal cross-learning for contextual bandits with unknown context distributions

We consider the problem of designing contextual bandit algorithms in the "cross-learning" setting of Balseiro et al., where the learner observes the loss for the action