Deconfounded Warm-Start Thompson Sampling with Applications to Precision Medicine

In many sequential decision-making problems, the decision-maker has access to contextual information that can guide personalized actions. Contextual Multi-Armed Bandit (CMAB) algorithms formalize this setting, extending traditional multi-armed bandits by incorporating side information (e.g., user attributes, environmental factors, or historical states) to inform arm selection [Lattimore and Szepesv ari, 2020]. This contextual adaptability makes CMABs especially valuable in dynamic, high-stakes applications like personalized healthcare, targeted advertising, and adaptive pricing. A growing body of literature explores leveraging observational (offline) data (collected from past interactions without active experimentation) to initialize and accelerate learning in CMABs [Ten-nenholtz et al., 2021, Hao et al., 2023]. Offline data provides a valuable source of information on context-arm-outcome relationships, which can substantially reduce the need for costly, risky, or even unethical online exploration in high-stakes settings. More importantly, observational data can play a critical role in identifying a sparse set ofinformative features that are most predictive of treatment responses. In fact, in high-dimensional settings, not all features equally influence the effectiveness of an action. Many features may be irrelevant or weakly correlated with the outcome, introducing noise and exacerbating the sample efficiency of online learning algorithms.