In this paper, we investigate the question: Given a small number of datapoints, for example N = 30, how tight can PAC-Bayes and test set bounds be made? For such small datasets, test set bounds adversely affect generalisation performance by withholding data from the training procedure. In this setting, PAC-Bayes bounds are especially attractive, due to their ability to use all the data to simultaneouslylearn a posterior and bound its generalisation risk. We focus on the case of i.i.d.

Neural Information Processing Systems http://nips.cc/

While therewardcompensation mechanism isunknown,the learner can adapt his (her) decision to the past reward feedback so as to maximize the sum of rewards.

We then give a practical approximation using neural networks anddemonstrate itsperformance andsampleefficiencyinpractice.

Insomework,functionapproximation scheme is adopted such that essential quantities for policy improvement, e.g.

Then, we consider sufficient assumptions under which learning good policies requires polynomial number of episodes.

On the technical side, we show that the logarithmic loss and an informationtheoretic quantity called thetriangular discriminationplay a fundamental role in obtaining first-order guarantees, and we combine this observation with new refinements tothe regression oracle reduction framework ofFoster and Rakhlin [29].

Note that inthis simple example, the state transition functionT and the reward functionr stillsatisfy theMarkovproperty.

O((|S ||A|+ T) H ( log ( 1/")/")). ItisalsoeasyV ( , )andQ ( , , )obtained algorithm.