Learning to Generalize from Sparse and Underspecified Rewards