Take Me Home: Reversing Distribution Shifts using Reinforcement Learning