To Switch or Not to Switch? Balanced Policy Switching in Offline Reinforcement Learning