Learning In-between Imagery Dynamics via Physical Latent Spaces