Policy-Gradient Methods for Planning