While therewardcompensation mechanism isunknown,the learner can adapt his (her) decision to the past reward feedback so as to maximize the sum of rewards.

We then give a practical approximation using neural networks anddemonstrate itsperformance andsampleefficiencyinpractice.

Insomework,functionapproximation scheme is adopted such that essential quantities for policy improvement, e.g.

Then, we consider sufficient assumptions under which learning good policies requires polynomial number of episodes.

On the technical side, we show that the logarithmic loss and an informationtheoretic quantity called thetriangular discriminationplay a fundamental role in obtaining first-order guarantees, and we combine this observation with new refinements tothe regression oracle reduction framework ofFoster and Rakhlin [29].

Note that inthis simple example, the state transition functionT and the reward functionr stillsatisfy theMarkovproperty.

There, areward function isdrawn from one of multiple possible reward models atthebeginning ofeveryepisode, buttheidentity ofthechosen rewardmodel is not revealed to the agent. Hence, the latent state space, for which the dynamics are Markovian, is not given to the agent. We study the problem of learning a near optimal policy for two reward-mixing MDPs. Unlike existing approaches that rely on strong assumptions on the dynamics, we make no assumptions and study the problem in full generality.

Nevertheless, despite theproliferation ofresearch onalgorithmic fairness inrecent years, veryfew methods exist that can handle multiclass classification tasks with non-binary sensitive attributes.