Physics-informed Value Learner for Offline Goal-Conditioned Reinforcement Learning