Moreover, weassumethatthedimensionsd and k are fixedthroughoutthispaper.

Neural Information Processing Systems http://nips.cc/

Recently,Sutton[23]proposed anewview on action selection.

Itisknown that policyevaluation has the interpretation of solving ageneralized Bellman equation. Inthispaper,wederivefinite-sample bounds foranygeneral off-policy TD-like stochastic approximation algorithm that solves for the fixedpoint of this generalized Bellman operator.

Each algorithm optimises its parameters with respect to an objective, such as Q-learning or policy gradient, that defines its semantics.

Assume addition H3andset k =O(1/ p k). Foranyn 1,