Recommendation System-based Upper Confidence Bound for Online Advertising

--In this paper, the method UCB-RS, which resorts to recommendation system (RS) for enhancing the upper-confidence bound algorithm UCB, is presented. The proposed method is used for dealing with non-stationary and large-state spaces multi-armed bandit problems. The proposed method has been targeted to the problem of the product recommendation in the online advertising. Through extensive testing with RecoGym, an OpenAI Gym-based reinforcement learning environment for the product recommendation in online advertising, the proposed method outperforms the widespread reinforcement learning schemes such as null -Greedy, Upper Confidence (UCB1) and Exponential Weights for Exploration and Exploitation (EXP3). I NTRODUCTION Online advertising is becoming increasingly popular and is the main motivation for the development of almost free internet platforms such as search engines, social networks, recruitment sites, multimedia contents (e.g., videos, images, musics, ...) sharing, etc. From the point of view of the internet users, the product recommendation on online advertising can be genuinely useful if it meets the real immediate needs of users. Instead of spending a lot of time and effort searching for a huge number of thousands or even millions of choices, most internet users will be quite satisfied if recommendation systems propose exactly what they need. Finding a good recommendation system, therefore, continues to be the goal of many studies [1], [2]. Online and offline approaches for learning optimal recommendation policies can be found in the literature.