Contextual Dynamic Pricing with Unknown Noise: Explore-then-UCB Strategy and Improved Regrets

Dynamic pricing is a fast-moving research area in machine learning and operations management. A lot of work has been done for this problem with known noise. In this paper, we consider a contextual dynamic pricing problem under a linear customer valuation model with an unknown market noise distribution F . This problem is very challenging due to the difficulty in balancing three tangled tasks of revenue-maximization, estimating the linear valuation parameter \theta_{0}, and learning the nonparametric F . To address this issue, we develop a novel {\it Explore-then-UCB} (ExUCB) strategy that includes an exploration for \theta_{0} -learning and a followed UCB procedure of joint revenue-maximization and F -learning.