The ability to combine known skills to create new ones may be crucial in the solution of complex reinforcement learning problems that unfold over extended periods. We argue that a robust way of combining skills is to define and manipulate them in the space of pseudo-rewards (or cumulants). Based on this premise, we propose a framework for combining skills using the formalism of options. We show that every deterministic option can be unambiguously represented as a cumulant defined in an extended domain. Building on this insight and on previous results on transfer learning, we show how to approximate options whose cumulants are linear combinations of the cumulants of known options. This means that, once we have learned options associated with a set of cumulants, we can instantaneously synthesise options induced by any linear combination of them, without any learning involved. We describe how this framework provides a hierarchical interface to the environment whose abstract actions correspond to combinations of basic skills. We demonstrate the practical benefits of our approach in a resource management problem and a navigation task involving a quadrupedal simulated robot.

General V alue Functions (GVFs) (Sutton et al., 2011) represent predictive knowledge in reinforcement learning.

Second, we should be able to also play chords.

(conclusion 1). (conclusion 2). Z contains and only contains exogenous noises w.r.t. " means source and " Based on Theorem 6, we can readily give proof to Theorem 2. Note that in our setting where " is equivalent to " Theorem 7 (Trek-separation for directed graphical models, Theorem 2.8 in [ We now show that Theorem 2 can also be proved by trek-separation theorem: Proof of Theorem 2 (another version). 's noise components that is not shared in Therefore, the direction between X and Y is unidentifiable. GIN( Z, Y) must hold, with solution ω .

Our results begin to elucidate the structure of learned representations in wide BNNs.

To this end, we examine the Edgeworth expansion for the partition function given by Eq.