Latent Adaptive Planner for Dynamic Manipulation