Frequently,we are also interested in the scores associated with the objects to deduce the intensity of the resulting preference.

We then give a practical approximation using neural networks anddemonstrate itsperformance andsampleefficiencyinpractice.

Insomework,functionapproximation scheme is adopted such that essential quantities for policy improvement, e.g.

Then, we consider sufficient assumptions under which learning good policies requires polynomial number of episodes.

On the technical side, we show that the logarithmic loss and an informationtheoretic quantity called thetriangular discriminationplay a fundamental role in obtaining first-order guarantees, and we combine this observation with new refinements tothe regression oracle reduction framework ofFoster and Rakhlin [29].

This paper focuses on MDPs with linearly realizable optimal Q-functionQ?.

O((|S ||A|+ T) H ( log ( 1/")/")). ItisalsoeasyV ( , )andQ ( , , )obtained algorithm.

There, areward function isdrawn from one of multiple possible reward models atthebeginning ofeveryepisode, buttheidentity ofthechosen rewardmodel is not revealed to the agent. Hence, the latent state space, for which the dynamics are Markovian, is not given to the agent. We study the problem of learning a near optimal policy for two reward-mixing MDPs. Unlike existing approaches that rely on strong assumptions on the dynamics, we make no assumptions and study the problem in full generality.

In online learning problems, exploiting low variance plays an important role in obtaining tight performance guarantees yet ischallenging because variances are often not known a priori. Recently, considerable progress has been made by Zhangetal.