Appendix A Experiments In this section, we demonstrate our maximum likelihood estimation (8)

In the first experiment, we compare the performance of the aforementioned four alternatives. As we generated instances to satisfy the full-rank condition, i.e., Assumption E.1, or if random perturbations are applied to the underlying LMAB model [8]). (recall Figure 3). Multi-Armed Bandits problem that has been extensively studied in literature ( e.g., see [ When the time-horizon is sufficiently long but finite e.g., if Regime switching bandits LMAB may be also seen as a special type of adversarial or non-stationary bandits ( e.g., [ The standard objective in non-stationary bandits is to find the best stationary policy in hindsight with unlimited possible contexts. We focus on significantly more general cases where there is no obvious way of clustering observations, e.g., when Note that this could still be in H A regime with large number of actions A .