Online Reinforcement Learning for Mixed Policy Scopes