Multimodal Policies with Physics-informed Representations