Domain-independent probabilistic planners input an MDP description in a factored representation language such as PPDDL or RDDL, and exploit the specifics of the representation for faster planning. Traditional algorithms operate on each problem instance independently, and good methods for transferring experience from policies of other instances of a domain to a new instance do not exist. Recently, researchers have begun exploring the use of deep reactive policies, trained via deep reinforcement learning (RL), for MDP planning domains. One advantage of deep reactive policies is that they are more amenable to transfer learning. In this paper, we present the first domain-independent transfer algorithm for MDP planning domains expressed in an RDDL representation. Our architecture exploits the symbolic state configuration and transition function of the domain (available via RDDL) to learn a shared embedding space for states and state-action pairs for all problem instances of a domain. We then learn an RL agent in the embedding space, making a near zero-shot transfer possible, i.e., without much training on the new instance, and without using the domain simulator at all. Experiments on three different benchmark domains underscore the value of our transfer algorithm. Compared against planning from scratch, and a state-of-the-art RL transfer algorithm, our transfer solution has significantly superior learning curves.

The paper proposes a method termed TransPlan, to use Graph Convolutional Networks to learn the relations defined by an RDDL description to learn neural network policies that can "transfer" to different MDP planning domain instances. The architecture combines several components including a state encoder, an action decoder, a transition transfer module and a problem instance classifier. Only the action decoder requires retraining for transfer and the paper shows how a different component in the architecture (transition transfer module) can be used to quickly retrain and get substantial gains in transfer to a new domain without any "real" interactions (zero-shot). Authors evaluate their performance on benchmark domains from IPPC 2014 and show substantial improvements over standard algorithms which do not leverage the structure offered by an RDDL description of the problem. The authors also a do a few ablations studies to find the relative importance of different components in their system.

Domain-independent probabilistic planners input an MDP description in a factored representation language such as PPDDL or RDDL, and exploit the specifics of the representation for faster planning. Traditional algorithms operate on each problem instance independently, and good methods for transferring experience from policies of other instances of a domain to a new instance do not exist. Recently, researchers have begun exploring the use of deep reactive policies, trained via deep reinforcement learning (RL), for MDP planning domains. One advantage of deep reactive policies is that they are more amenable to transfer learning. In this paper, we present the first domain-independent transfer algorithm for MDP planning domains expressed in an RDDL representation.