Inrecentyears,quantitativestatistical and algorithmic treatments of these formulations have produced insights into modern computational methods-resulting innovelapproaches todifficult, open problems.

In the reinforcement learning context, anOption means a temporally extended sequence of actions [30],andisregarded asuseful formanypurposes, such asspeeding uplearning, transferring skills across domains, and solving long-term planning problems.

We focus on the setting where firms are not aware of demand functions and how reference prices areformed but haveaccess to an oracle that provides a measure of consumers' responsiveness to the current posted prices.

Online retailing has become increasingly popular over the last decades [17, 28, 52]. The way of product ranking is the crux for online retailers because it determines the consumers' shopping behaviors [17] and thus influences the retailers' revenue [20, 49]. For instance, the probability of consumers' purchasing from a firm or clicking an advertisement is strongly related to the display order[8,3,33].

To model the demand as a function of price and context, the existing literature either adopts a parametric model or a non-parametric model.

Popular algorithms suchasUCB[4,5]andThompson sampling [3,34]typically explorethearms sufficiently and as more evidence is gathered, converge to the optimal arm.

Wedesign a novel truncation operation to further reduce the computational complexity of theproposed algorithms.

Inthesecondexample, vacuuming the floors of a house has certain risks, but the consequences of optimizing the wrong rewardfunction arearguably muchlesssignificant.

We propose a randomized dynamic pricing policy based on a variant of the Online Newton Stepalgorithm (ONS)thatachievesaO(d T log(T))regretguarantee underan adversarial arrival model.