Learning to Generalize for Sequential Decision Making