MANGA: Method Agnostic Neural-policy Generalization and Adaptation